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FOREWORD 


The  Naval  Training  Equipment  Center  Is  pleased  to  sponsor  the  Eleventh  NAVTRAEQUIPCEN/ 1 ndus try 
Conference  with  the  theme,  "New  Horizons  for  Simulation." 

The  major  growth  of  our  industry  over  the  past  decade  can  be  attributed  to  Inherent  economic 
and  ecologlc  advantages  coupled  with  significant  technological  advances  which  have  enabled 
improved  fidelity  and  new  applications  of  simulation  technology  and  training  methodology.  In 
particular,  it  has  been  through  exploitation  of  modern  cost-effective  digital  computer  systems 
that  these  achievements  have  been  possible.  This  same  technology  will  permit  the  merging  of 
curriculum  and  equipment  into  computer-based  training  systems  using  the  Instructional  Systems 
Development  process. 

The  annual  NAVTRAEQUIPCEN/ Industry  Conference  is  an  Important  event  In  continuing  the  frank 
and  open  dialogue  that  Is  vital  to  meeting  the  challenges  of  current  and  future  training  system 
development.  The  constructive  and  creative  environment  of  this  Conference  serves  as  a forum 
for  establishing  and  maintaining  a rapport  throughout  the  training  community. 

The  papers  published  in  these  proceedings  are  from  government,  industry  and  universities 
and  they  represent  all  phases  of  the  process  from  training  needs  identification  to  life-cycle 
logistics  support.  These  proceedings  will  play  an  Important  role  in  the  decision-making 
process  of  exploring  various  concepts  and  alternatives  to  satisfy  specific  training  system 
requirements. 
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INTRODUCTION  TO  THF  CONFERENCE 


G.  V.  Amlco 

Director  of  Engineering 
Naval  Training  Equipment  Center 

The  theme  of  this  year's  conference,  "New  Horizons  for  Simulation,"  permits  us  to  assess 
the  progress  which  has  been  made  over  the  last  decade  and  to  forecast  the  advancement  which 
will  be  made  within  the  next  decade.  I would  like  to  briefly  cover  both  the  mid-  and  long- 
range  horizons  for  simulation  technology  as  It  applies  to  training. 

The  mid-range  horizon  will  be  Influenced  by  the  continued  advancements  being  made  In  the 
computer  field  and  will  center  around  the  development  of  training  systems.  These  training 
systems  will  encompass  a wide  range  of  Instructional  media,  primarily  computer  based  or 
controlled  systems,  to  support  major  weapon  systems.  In  consonance  with  this  concept,  there 
will  be  an  increase  in  the  use  of  computer  controlled  and  managed  instructional  systems. 

These  systems  will  provide  the  self-pacing,  adaptive  and  self-evaluating  features  of  the 
learning  process  that  are  geared  to  the  behavioral  requirements  of  the  operational  systems. 
Reductions  in  instructor  requirements  will  be  an  important  feature  of  these  systems. 

As  the  complexity  of  operational  equipment  increases  through  the  use  of  onboard  computers 
with  their  associated  control  and  display  systems,  the  cost  of  the  operational  equipment 
necessary  to  support  operator  and  team  trainings  at  multiple  sites  becomes  prohibitive.  This 
cost  factor  alone  will  lead  to  training  equipment  which  can  be  directly  interfaced  with  the 
operational  system  in  a matter  of  hours.  The  pierside  concept  for  surface  ships  has  already 
been  proved  by  programs  which  have  been  undertaken  during  the  last  few  years. 

The  mid-range  horizon  will  also  include  the  development  of  multi-window,  multi-channel 
computer  image  generation  systems  not  only  for  aircraft,  but  also  for  surface  ships  and  land 
based  vehicles. 

The  high  cost  associated  with  using  operational  equipment  to  support  all  phases  of 
specialized  maintenance  training  will  lead  to  the  use  of  effective  combinations  of  courseware 
and  simulation  hardware  for  many  portions  of  the  curriculum.  These  maintenance  training 
systems  will  provide  more  effective  training  at  a substantially  lower  cost. 

The  longer  range  horizon  envisions  systems  that  will  be  developed  in  the  latter  part  of 
the  next  decade.  The  direction  now  being  established  for  the  mid-range  will  continue.  The 
training  system  approach  will  have  been  fully  implemented  for  new  weapon  systems.  These 
training/instructional  centers  will  employ  fully  automated  curricula  under  the  control  of  a 
mul ti -computer  complex  capable  of  monitoring  performance  against  established  standards  for 
maintainers,  operators  and  combat  teams.  These  same  training  concepts  will  also  be  applied 
to  operational  subsystems  with  critical  and  complex  training  tasks. 

The  pierside  design  will  move  closer  to  the  embedded  training  system  concept.  The 
increased  use  of  computers  in  operational  systems  will  simplify  the  connection  of  the 
environmental  generator  and  instructor  controls  to  the  operational  system.  This  connection 
may  reside  within  the  program  of  the  operational  computer  or  in  a satellite  computer.  Advances 
in  simulation  and  computer  technology  will  make  almost  every  vehicle  serve  its  own  training 
needs.  Through  data  links,  the  combat  teams  in  a number  of  vehicles  can  be  trained  to  react 
to  various  multi -threat  scenarios. 

High-performance,  wide-angle,  visual  systems  using  computer-generated  imagery  will  find 
wide  application  in  many  training  areas  where  visual  perception  is  a critical  pilot/operator 
function.  Major  advancements  will  be  in  the  areas  of  display  technology. 

Specialized  maintenance  training  facilities  that  Integrate  the  operational  equipment 
maintenance  manual/technical  data,  parts  identification  and  supply  functions  with  the 
diagnostic/corrective  maintenance  procedures  will  rely  heavily  on  simulation  technology  and 
computer  based  Integrated  courseware. 
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The  major  advancements  which  have  been  forecast  rely  heavily  on  the  continuation  of  current 
trends  in  computer  development,  namely  increased  capacity/ speed  at  a reduced  cost.  The 
Achilles'  heel  associated  with  these  projections  is  in  the  area  of  software  engineering.  To 
date,  software  engineering  problems  have  plagued  every  major  trainer  development.  Although  the 
problem  is  universally  recognized,  less  than  satisfactory  measures  have  been  taken  to  remove 
or  allevi-ate  the  problem.  If  this  deficiency  is  not  corrected,  the  bright  horizon  for  training 
systems  will  be  dark  and  clouded. 

The  new  horizon  for  simulation  is  indeed  bright.  Its  principal  feature  will  be  the 
effective  integration  of  hardware  and  courseware  into  training  systems.  These  training  systems 
will  encompass  a wide  range  of  curriculum  and  media  to  satisfy  a variety  of  operator/maintainer 
training  objectives.  Both  initial  and  refresher  training  needs  will  be  accommodated.  This 
conference  will  permit  us  to  examine  that  horizon,  discuss  alternative  solutions,  and  develop  an 
optimum  training  strategy;  a strategy  to  improve  operational  readiness  as  its  cornerstone. 
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NAVAL  AVIATION  INSTRUCTIONAL  SYSTEMS  DEVELOPMENT 


CDR  JOSEPH  F.  FUNARO 
Deputy  for  Training  Systems  Development 
Naval  Training  Equipment  Center 
and 

OR.  B.  E.  MULLIGAN 
Department  of  Psychology 
The  University  of  Georgia 


Growth  In  the  Inherent  complexity  of  airborne 
weapons  systems  over  the  last  thirty-odd 
years  has  been  paralleled  by  the  emergence  of 
an  equally  complex  problem  - the  design  of 
large-scale  cost-effective  instructional 
systems.  Until  recently,  the  Navy's  response 
to  this  problem  has  been  much  the  same  as  the 
other  armed  services  and  industry.  The  con- 
cept of  systems  analysis  was  embraced  as  an 
operational  strategy  and  the  principles  of 
applied  psychology  and  educational  technology 
were  transformed  Into  procedural ized  method- 
ologies for  training  program  development. 

The  result  was  called  the  Systems  Approach  to 
Training  (SAT). 

The  principles  underlying  SAT  were  sound,  but 
the  approach  suffered  a fundamental  weakness. 
It  lacked  a management  system  capable  of  di- 
recting multiple  applications  of  SAT,  i.e., 
a general  control  process  that  would  effect  a 
uniform  application  of  SAT  across  training 
programs  developed  for  different  weapons 
systems. 

By  the  mid-1970s  the  Naval  Training  Equipment 
Center  (NAVTRAEQUIPCEN),  acting  under  the 
auspices  of  the  Naval  Air  Systems  Coirmand 
(NAVAIRSYSCOM),  had  undertaken  the  task  of 
constructing  a generally  applicable  model  for 
Instructional  Systems  Development  (iSD).  The 
ISO  process  was  divided  into  five  major 
phases  of  activity:  analysis,  design,  de- 
velopment, implementation,  and  quality  con- 
trol. The  activities  to  be  carried  out  with- 
in each  major  phase  were  identified  and  fur- 
ther developed  Into  logically  coherent  pro- 
cedures that  made  explicit  the  key  decision 
points,  specific  objectives,  and  end  products 
required  at  each  stage  in  the  process.  The 
resulting  standardization  of  the  ISD  process 
made  possible  effective  management  control 
throughout  the  successive  stages  of  each  de- 
velopmental project,  and  it  permitted  cross- 
project comparisons  essential  to  further  re- 
finement and  generalization  of  the  process. 

The  NAVAIRSYSCOM/NAVTRAEQUIPCEN  model  of  the 
ISD  process  has  been  widely  applied  in  the 
development  of  a variety  of  training  programs 
throughout  Naval  Aviation,  and  the  model  has 
undergone  several  revisions  in  response  to 
feedback  from  these  applications.  As  a 
management  tool,  the  model  has  proven  its 
value,  but  it  continues  to  be  evaluated  as 


experience  accumulates.  The  present  formula- 
tion of  the  model  Is  described  nontechnically 
in  the  paragraphs  that  follow. 

Whether  the  model  Is  applied  to  a newly 
emerging  weapons  system,  or  to  an  existing 
system,  the  ISD  process  always  begins  with  a 
problem  analysis.  Generally,  the  ISD  process 
is  initiated  by  a documented  indication  of  a 
need  for  a new  training  program  or  a revi- 
sion of  an  existing  system.  This  initial 
documentation  identifies  the  nature,  scope, 
and  criticality  of  the  training  project.  If 
it  is  found  to  be  of  sufficient  magnitude  to 
warrant  further  consideration,  an  in-depth 
problem  analysis  is  planned  by  the  NAVTRA- 
EQUIPCEN. After  the  plan  has  received  the 
approval  of  the  NAVAIRSYSCOM,  and  the  lines 
of  comnunication  have  been  established  among 
the  various  participating  Navy  organizations, 
a full-scale  problem  analysis  is  launched 
under  the  direction  of  the  NAVTRAEQUIPCEN. 

The  primary  objective  of  the  problem  analysis 
is  to  establish  the  precise  nature  of  the 
training  effort  that  will  be  required  to 
achieve  an  effective  instructional  program. 

In  the  case  of  an  existing  training  program, 
every  aspect  of  the  system  is  examined  and 
evaluated  In  order  to  identify  the  needed 
revisions  and  the  appropriate  strategies  for 
making  them.  The  analysis  would  include 
everything  from  Instructional  syllabi, 
training  materials  and  devices,  tests  and 
student  attitudes  to  management  efficiency 
and  instructional  goals  of  the  existing 
system. 

In  the  case  of  an  emerging  weapons  system, 
the  analysis  is  directed  toward  a preliminary 
determination  of  the  tasks  required  to  oper- 
ate the  system,  the  kind  of  training  program, 
materials,  and  devices  that  will  optimize 
student  learning  of  the  required  tasks,  and 
the  structure  of  the  Navy  organization  that 
will  be  needed  to  develop,  implement,  and 
manage  the  instructional  system.  In  the 
case  of  either  an  existing  or  an  emerging 
system,  the  problem  analysis  identifies  the 
personnel,  facilities,  equipment,  time,  and 
costs  required  for  development  and  implemen- 
tation of  the  needed  training  program.  This 
information  is  evaluated  relu'  'e  to  program 
goals  and  assets,  and  a Problem  Analysis 
Report  (PAR)  is  prepared.  If  thz  projected 
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goals  and  requirements  of  the  needed  training 
program,  as  documented  in  the  PAR,  are 
approved  by  the  NAVAIRSYSCOM,  the  development 
of  a program  master  plan  Is  Initiated. 

The  program  master  plan  (PMP)  serves  as  a 
tool  for  managing  and  coordinating  the  ISD 
project,  and  it  incorporates  the  information 
contained  in  the  previous  problem  analysis 
together  with  other  Information  such  as  the 
latest  funding  analyses  available  and  costs 
data  from  previous  ISD  projects.  The  model 
is  also  taken  into  account  in  the  PMP  since 
the  model  delineates  the  stage-by-stage  pro- 
gression of  ISD  and  stipulates  the  products 
generated  at  each  stage.  All  this  informa- 
tion is  integrated  by  the  NAVTRAEQUIPCEN  and 
the  PMP  is  formulated. 

The  plan  states  the  objectives  of  the  pro- 
posed ISD  program,  and  the  procedures  by 
means  of  which  those  objectives  can  be  real- 
ized. The  major  milestones  of  the  ISD  proj- 
ect are  specified,  and  a method  is  provided 
for  tracking  the  various  stages  of  the  de- 
velopment process.  The  resources  and  facil- 
ities required  for  the  ISD  project  are  in- 
ventoried together  with  the  sources  committed 
to  making  them  available.  The  plan  also 
stipulates  the  organizational  roles  and  re- 
sponsibilities of  participating  organizations 
and  personnel , and  provides  a system  for 
their  coordination.  The  plan  even  designates 
the  specific  jobs  to  be  performed  throughout 
the  ISD  project,  and  indicates  when  each  job 
is  to  be  performed,  and  by  whom.  The  plan 
also  identifies  potential  funding  sources, 
gives  cost  estimates  for  each  of  the  suc- 
cessive stages  of  the  ISD  project,  offers 
various  procurement  strategy  options,  and  de- 
scribes the  materials  needed  to  carry  out 
appropriate  procurement  procedures.  Thus, 
the  PMP  serves  as  the  primary  management  tool 
for  directing  the  development  of  a large- 
scale  instructional  system.  If  the  PMP  is 
approved  by  the  NAVAIRSYSCOM,  the  project  may 
then  move  ahead  Into  the  more  detailed  task 
analysis  stage  where  the  real  job  of  behav- 
ioral specification  and  instructional  design 
begins. 

It  is  in  the  analysis  phase  of  ISD  where  the 
techniques  of  modern  psychology  and  instruc- 
tional technology  are  applied  most  intensive- 
ly in  the  model.  The  analysis  phase  begins 
with  the  question,  "What  are  the  tasks  that 
must  be  performed  in  order  to  operate  the 
system  in  question,  and  under  what  conditions 
must  these  tasks  be  performed?"  The  answer 
to  this  question  is  obtained  through  a task 
analysis  of  the  operational  system. 

First,  system  operation  is  partitioned  into 
the  major  job  responsibilities  required  for 
each  mission  phase.  Then,  each  responsi- 
bility area  is  further  analyzed  to  identify 
the  primary  task  components  that  must  be 


performed.  Each  task  component  Is  described 
In  a detailed  and  standardized  fashion.  A 
task  description  states  precisely  the  condi- 
tions under  viilch  the  task  is  performed,  the 
actions  that  compose  the  performance,  and  the 
particular  outcomes  of  the  performance.  This 
emphasis  on  task  identification  and  descrip- 
tion is  based  on  the  psychological  principle 
that  "the  more  accurately  a behavior  can  be 
specified,  the  more  efficiently  it  may  be 
trained."  Thus,  task  analysis  generates  task 
specifications,  i.e.,  descriptive  statements 
of  the  conditions,  actions,  and  outcomes  that 
compose  each  task  component.  By  organizing 
these  task  descriptions  according  to  respon- 
sibility areas  and  mission  phases,  an  accurate 
picture  required  to  perform  a job  is  obtained. 

The  task  analysis  and  listing  process  is 
usually  carried  out  by  a team  of  subject 
matter  experts  (SMEs),  Navy  personnel  experi- 
enced in  the  operation  of  the  weapons  system 
under  consideration,  who  have  been  trained  in 
ISD  methodology  by  a behavioral  specialist. 
Together,  the  specialist  and  SMEs  form  the 
ISD  team  responsible  for  producing  the  ini- 
tial  task  listing.  To  ensure  that  the  task 
descriptions  are  accurate,  and  that  the  task 
listing  is  complete  and  properly  organized, 
an  independent  group  of  experienced  fleet 
personnel  evaluates  the  work  of  the  original 
ISD  team.  They  also  estimate  the  frequency 
and  criticality  of  performance  of  the  indi- 
vidual tasks.  This  step  in  the  ISD  process 
is  referred  to  as  task  validation,  and  it  may 
result  in  a revision  of  the  original  listing. 

As  a means  of  standardizing  the  validation 
procedure,  the  ISD  team  prepares  a question- 
naire which  is  used  by  fleet  personnel  to 
evaluate  the  task  listing. 

Now  that  the  tasks  have  been  behaviorally 
specified  and  functionally  organized,  the 
validated  task  listing  is  subjected  to  another 
kind  of  evaluation.  Each  task  in  the  listing 
is  examined  systematically  by  the  ISD  team  in 
order  to  determine  the  level  of  training  that 
it  will  require. 

Depending  on  the  entry-level  skills  of  the 
new  Fleet  Readiness  Squadron  (FRS)  trainees, 
as  compared  with  the  standards  of  acceptable 
performance  for  individual  tasks,  the  ISD 
team  will  assign  each  task  to  one  of  five 
different  training  categories.  These  five 
categories  are;  full-scale,  review-only, 
familiarization-only,  deferred,  and  no- 
training required.  For  example,  if  a given 
task  has  to  be  performed  frequently,  and  if 
its  performance  standard  exceeds  the  entry- 
level  skills  of  trainees,  the  task  would  be 
scheduled  for  full-scale  FRS  training,  espe- 
cially if  correct  performance  on  the  task  weie 
critical.  On  the  other  hand,  even  a high-fre- 
quency task  might  be  classified  as  requiring 
no  training  if  its  performance  were  already 
well  within  entry-level  skills  of  new  trainees. 
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As  a result  of  this  careful  procedure  of  task 
selection,  a major  benefit  Is  achieved.  Re- 
sources  and  time  are  not  wasted  on  unneces- 
sary training,  and  tasks  that  are  essential 
to  competent  performance  are  not  overlooked 
in  the  training  program.  Rather,  the  ap- 
proach taken  i*  to  assign  each  task  to  just 
that  level  of  training  which  is  necessary  to 
assure  that  Its  performance  will  at  least 
meet  the  operational  standard. 

At  this  point  the  tasks  selected  for  FRS 
training  are  reviewed  to  determine  which  of 
them  must  be  trained  in  either  real,  or  simu- 
lated, operational  environments.  Tasks  that 
require  live  enactment  or  perceptual -motor 
skills  involving  realistic  visual,  auditory, 
motion,  etc.,  cues  are  designated  for  "hands- 
on"  media  training.  This  group  of  tasks  is 
"Further  divided  into  two  categories;  those 
that  must  be  trained  only  in  actual  flight 
conditions  (or  only  on  actual  operational 
equipment),  and  those  that  can  be  trained 
under  synthetic  operational  conditions.  It 
is  the  latter  group  of  "hands-on"  tasks  that 
are  of  interest  at  this  stage  of  the  ISD 
process. 

Analysis  of  the  conditions  and  standards  as- 
sociated with  these  tasks  enables  the  ISD 
team  to  arrive  at  a definition  of  the  kinds 
of  training  devices  that  will  be  needed  for 
simulated  "hands-on"  training.  The  attempt 
here  is  not  only  to  select  the  most  suitable 
training  medium  for  each  task,  but  to  iden- 
tify as  early  as  possible  those  media  which 
require  long  lead  time  for  design  and  produc- 
tion. This  permits  parallel  development  of 
both  the  training  program  and  the  synthetic 
training  media,  thus  avoiding  costly  delays 
in  implementation. 

The  armed  services  have  long  recognized  that 
training  devices  are  less  expensive  to  use  as 
training  media  than  actual  operational  sys- 
tems. The  savings  in  personnel  time  and  fuel 
consumption  alone  are  sufficient  to  warrant 
their  use  in  training  programs.  But  it  has 
also  been  recognized  that  synthetic  devices 
enable  the  training  of  many  procedures  (such 
as  those  involving  emergencies)  that  could 
not  otherwise  receive  any  degree  of  Instruc- 
tional attention.  However,  the  approach 
taken  by  the  model  extends  the  advantages  of 
training  devices  even  beyond  this  level  of 
utility. 

In  the  model,  behavioral  objectives  of  the 
instructional  program  control  training  de- 
vice specification.  This  concept  Is  in 
marked  contrast  with  the  old  notion  that  a 
training  device  should  simulate  the  opera- 
tional system  as  closely  as  possible.  To  the 
surprise  of  many,  a training  device  optimally 
designed  to  meet  behavioral  objectives  may 
Involve  considerably  less  simulation  than 
those  produced  by  the  traditional  approach  to 
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training.  Furthermore,  where  device  specifi- 
cations are  governed  by  behavioral  objectives, 
the  devices  are  not  only  more  effective  as 
training  media,  but  they  may  be  less  expensive 
to  produce  and  operate.  Since  just  that  de- 
gree of  simulation  that  is  necessary  to  ade- 
quately train  tasks  is  Incorporated  into  a 
device,  the  expense  of  unnecessary  simulation 
is  avoided.  Training  devices  designed  to  the 
specifications  generated  from  the  ISD  approach 
are  thus  effective  for  minimal  cost.  It  is 
significant  that  these  devices  are  designed  to 
be  an  integral  part  of  the  overall  training 
program  to  serve  as  instructional  media  with 
specific  training  objectives.  This  assures 
an  extensive  utilization  of  available  devices 
and,  consequently,  the  realization  of  a higher 
return  on  the  Navy's  investment. 

What  we  have  seen  up  to  this  point  in  the  ISD 
process  is  a systematic  analysis  of  the  be- 
havior necessary  to  operate  a weapons  system. 
For  each  operator  position,  the  necessary  be- 
havior is  broken  down  into  its  component 
tasks,  and  these  tasks  are  organized  into  the 
functional  units  that  occur  within  each  phase 
of  a mission.  The  task  listing  is  independ- 
ently validated,  and,  if  necessary,  then  re- 
vised. The  conditions,  outcome,  frequency, 
standards,  and  criticality  of  performance  of 
each  task  is  specified.  Tlese  factors,  to- 
gether with  an  assessment  of  entry-level 
skills  of  new  trainees,  provides  the  data 
needed  to  select  the  tasks  to  receive  FRS 
training  and  to  designate  the  type  and  level 
of  training  required  for  each  task.  "Hands- 
on"  tasks  that  can  be  best  trained  in  syn- 
thetic devices  are  identified,  and  this  en- 
ables an  early  specification  of  the  training 
devices  that  will  become  an  integral  part  of 
the  training  program. 

While  the  analyses  of  behavior  carried  out 
thus  far  are  adequate  for  task  specification 
and  selection,  an  even  finer  analysis  is 
necessary  to  determine  the  nature  of  the 
behavioral  objectives  that  the  training  pro- 
gram must  be  designed  to  achieve. 

The  distinction  between  tasks  and  behavioral 
objectives  is  fundamental  to  the  ISD  method- 
ology, for  It  is  the  objectives  that  control 
the  detailed  aspects  of  instructional  design. 
It  may  be  said  that,  whereas  tasks  are  what  a 
person  must  do  to  operate  a system,  behavioral 
objectives  are  vrfiat  a training  program  must 
achieve  to  produce  competent  task  performance. 

The  behavior  required  to  perform  a complex 
task  always  contains  a nunber  of  component 
skills,  concepts,  decision-making  strategies, 
etc.  Consequently,  whole-task  performance 
cannot  be  trained  satisfactorily  until  its 
more  basic  components  have  been  learned.  For 
example,  the  use  of  an  on-board  computer  to 
calculate  the  time-of-arrival  at  some  distant 
destination  requires  that  the  operator  not 
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only  possess  the  fundamental  computer  skills, 
but  he  must  also  have  knowledge  of  the  more 
general  navigational  principles  and  tech- 
niques, as  veil  as  the  basic  mathematical 
skills  essential  for  numerical  computation. 

Use  of  the  on-board  computer  and  applications 
of  navigational  principles  could  not  be 
taught  effectively  if  the  trainee  did  not 
first  possess  the  elemental  mathematical  and 
computational  skills. 

So  by  breaking  down  a task  into  its  funda- 
mental components,  and  by  comparing  these 
with  the  skills  already  present  in  the  be- 
havior of  the  new  trainees,  it  is  possible  to 
identify  the  particular  behavioral  components 
of  a task  that  must  be  individually  trained. 
These  task  components  become  the  behavioral 
objectives  to  be  achieved  by  the  training 
program. 

This  analysis  of  the  behavior  required  to 
perform  a task,  however,  only  yields  a set  of 
target  behaviors  that  the  ISD  program  must  be 
designed  to  produce  in  trainees  at  criterion 
proficiency  levels  under  specified  conditions. 
These  target  behaviors,  or  terminal  objec- 
tives, may  be  many  steps  removed  from  the 
basic  entry-level  skills  of  the  trainees. 

Thus,  the  behavioral  components  intermediate 
between  entry-level  skills  and  terminal  ob- 
jectives also  must  be  enumerated. 

Again,  the  driving  principle  behind  this  pro- 
gressively more  detailed  analysis  of  behavior 
is  to  make  explicit  that  v^iich  must  be 
trained.  Intermediate  behaviors  are  prerequi- 
sites for  task  performance.  The  relation- 
ships among  intermediate  behaviors  and  target 
behaviors  form  an  organizational  hierarchy 
shaped  like  a pyramid  with  the  behavioral  ob- 
jectives located  at  the  top.  Behaviors  listed 
at  each  level  of  the  objectives  hierarchy  are 
always  prerequisite  and  essential  to  perform- 
ance  of  the  behaviors  listed  at  higher  levels 
in  the  hierarchy. 

Construction  of  objective  hierarchies  is  a 
crucial  step  in  the  ISD  process  for  several 
reasons.  First,  the  construction  procedure 
helps  to  ensure  that  the  ISD  team  will  not 
overlook  any  important  intermediate  behaviors. 
Second,  the  hierarchical  organization  of  the 
intermediate  behaviors  leading  to  each  behav- 
ioral objective  shows  the  sequential  order  in 
which  these  behaviors  should  be  learned.  And, 
third,  the  entire  subject  matter  content  of 
the  instructional  program  is  delineated  in  the 
hierarchical  organization  of  the  behavioral 
objectives.  Thus,  objective  hierarchies  pro- 
vide a complete  picture  of  the  diversity  of 
behaviors,  and  their  Interrelationships,  that 
must  be  encompassed  by  the  instructional 
system. 

Now  that  the  question  of  what  to  teach  has 
been  definitely  answered,  the  next  question 


to  be  addressed  is  how  to  teach  it  most  ef- 
fectively? An  answer  to  this  question  rust 
be  obtained  for  each  behavioral  objective, 
and  each  answer  must  identify  both  the  method 
and  medium  for  Instruction  that  will  optimize 
student  learning  of  the  behavior. 

Knowing  what  to  teach  is  usually  not  synony- 
mous with  knowing  how  best  to  teach  it.  For 
this  reason,  the  model  provides  the  ISD  team 
with  a decision-making  procedure  which  can  be 
used  In  a straightforward  manner  to  select 
methods  and  media  given  certain  inputs  to  the 
process.  The  ISb  team  first  determines  the 
resources  that  will  be  available  for  instruc- 
tion, i.e.,  the  funds,  personnel,  and  facili- 
ties. These  resources  set  limits  on  the  range 
of  methods  and  media  from  which  the  team  may 
select.  After  this  is  established,  the  ISD 
team  looks  at  the  subject  matter  content  of 
each  behavioral  objective  to  ascertain  the 
kind  and  level  of  learning  involved,  the 
level  of  competency  students  will  be  expected 
to  obtain,  the  kinds  of  interactions  with  in- 
structors and  materials  that  will  be  needed  to 
substantiate  and  motivate  learning,  the  speci- 
ficity and  source  of  response-contingent  feed- 
back necessary  for  self-corrective  learning, 
and  the  characteristics  of  information  dis- 
plays essential  for  effective  presentation  of 
the  subject  matter.  When  this  information 
has  been  obtained,  the  ISD  team  is  ready  to 
begin  the  decision-making  process  that  will 
result  in  the  optimal  choices  of  an  available 
method  and  medium  for  each  particular  subject 
matter  under  consideration. 

Advances  in  the  application  of  electronics  to 
the  development  of  instructional  media  has  had 
a profound  influence  on  educational  technolo- 
gy. This  is  especially  evident  in  some  of  the 
applications  of  computer  science.  Consider, 
for  example,  the  sophisticated  visual  and 
auditory  displays  now  utilized  in  modern  in- 
structional media  and  simulation  devices,  the 
advanced  computer  programs  that  allow  for 
student  interaction  and  provide  response -con- 
tingent feedback,  and  the  many  kinds  of  infor- 
mation processing  systems  that  extend  the 
limits  on  man's  memory  and  thinking  capabili- 
ties. 

No  longer  must  the  quality  of  instruction  be 
invested  primarily  in  the  expertise,  ability 
to  communicate,  and  motivating  influence  of 
the  traditional  Instructor.  The  diversity  of 
methods  and  media  existing  today  permit  the 
instructional  designer  to  choose  the  one  that 
is  most  appropriate  for  each  behavior  to  be 
trained.  Furthermore,  the  new  methods  and 
media  rely  less  on  the  instructor  operating 
as  a lecturer,  motivator,  and  evaluator.  In 
ISD  programs,  Instructional  presentation  and 
evaluation  Is  individualized,  and  motivation 
is  maintained  through  reinforcing  contingen- 
cies built  Into  the  progression  of  events  each 
student  encounters  as  he  moves  through  the 
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program.  This  shifts  the  responsibilities  of 
Instructors  and  students  from  those  of 
lecturer  and  pupil  to  those  of  guide  and 
learner.  The  respective  role  of  each  Is  en- 
hanced, as  Is  their  motivation  to  perform 
well.  The  new  responsibilities  of  Instruc- 
tors and  students  In  the  training  program  are 
largely  due  to  the  methods  and  media  selected 
for  use,  but  they  are  also  dependent  upon  the 
way  Instructional  courses  are  developed  and 
sequenced. 

The  sequence  of  Instructional  experiences 
through  which  the  students  move  In  an  ISD 
training  program  is  stipulated  in  a course 
syllabus,  a kind  of  map  In  which  behavioral 
objectives  are  organized  into  lesson  se- 
quences. A given  course  is  divided  into  major 
units  of  Instruction,  and  each  unit  consists 
of  a number  of  lessons  which  are  further 
broken  down  into  Individual  segments.  These 
divisions  of  a course  are  sequenced  to  lead 
onto  each  other  such  that  the  student  is 
moved  in  a steady  path  from  lower  to  higher 
order  knowledges  and  skills. 

The  progression  of  learning  experiences  Is 
arranged  so  that  the  student's  knowledge  and 
skills  build  in  Increments  small  enough  to 
never  overextend  the  students'  capability, 
but  large  enough  to  ensure  an  optimal  rate 
of  progression  from  entry-levels  to  the  more 
complex  job-levels  of  performance.  This  kind 
of  arrangement  is  designed  to  prevent  failure 
at  one  point  in  a course  due  to  incomplete 
experience  gained  at  previous  points  in  the 
course.  At  each  stage,  the  student  has  all 
the  experience  he  needs  to  proceed  success- 
fully to  the  next  stage. 

Instruction  programmed  in  this  fashion  has 
proved  to  be  a far  more  efficient  approach 
than  the  traditional  one  because  it  not  only 
reduces  failure  to  a minimum,  but  It  also 
produces  a high  level  of  student  motivation. 
When  effort- to- learn  results  in  success,  this 
feedback  motivates  the  student  to  continue. 
Thus,  the  sequential  ordering  of  learning 
experiences  into  well -designed  course  syllabi 
is  a crucial  aspect  of  the  ISD  process.  It  is 
at  the  stage  of  syllabi  development  that  the 
ISD  team  moves  from  behavioral  analysis  to  In- 
structional design,  Incorporating  the  informa- 
tion contained  In  the  objective  hierarchies 
into  a framework  designed  to  maximize 
learning. 

The  transformation  from  objective  hierarchies 
to  course  syllabi  is  nearly  as  complicated  as 
it  is  Important,  and  ISD  team  members  require 
some  training  and  expert  guidance  In  this 
process.  Essentially,  the  question  they  must 
answer  Is  "into  what  sequences  must  the  behav- 
ioral objectives  be  ordered?,"  and  this  ques- 
tion must  be  answered  for  each  behavioral 
objective. 


Usually,  the  position  of  a behavioral  ob- 
jective In  Its  hierarchy  will  determine  Its 
sequential  position  In  a course  syllabus. 
Objectives  located  at  the  bottom  of  a hier- 
archy are  more  elementary  than  those  above  It 
and,  thus,  should  be  taught  first.  However, 

If  all  low-level  objectives  were  taught  before 
moving  up  the  hierarchy,  the  student  would 
tend  to  become  bored  and  possibly  even  forget 
some  of  the  Information  he  has  already  learned 
because  he  would  not  have  had  an  opportunity 
to  apply  It.  So,  this  problem  Is  avoided  by 
Introducing  "hands-on"  experience  into  the 
syllabus  as  soon  as  possible.  Generally, 
this  can  be  accomplished  by  limiting  succes- 
sive sets  of  objectives  to  vertical  legs  of 
the  hierarchy.  In  other  words,  the  designer 
starts  with  the  lowest  objective  on  one  verti- 
cal leg  and  moves  up  to  the  point  where  a 
"hands-on"  objective  is  encountered.  The 
latter  may  be  anything  from  a familiarization 
exercise  in  a trainer  to  an  actual  flight  in 
an  aircraft.  This  type  of  sequence  cycles 
the  student  from  purely  ground  school  type 
situations  to  equipment  exercises,  then  back 
to  ground  school  for  more  basics  followed  by 
further  equipment  exercises. 

This  type  of  cycling  maintains  the  integrity 
of  the  objectives  hierarchy  while  allowing  the 
student  to  practice  his  newly  acquired  skills 
as  soon  as  possible.  In  this  way,  a small 
portion  of  the  syllabus  is  encountered, 
learned,  and  practiced  before  the  student 
moves  on.  Those  objectives  requiring  actual 
flight  in  the  aircraft  are  preceded  by  objec- 
tives of  a more  elemental  nature  that  are 
practiced  in  a trainer  or  simulator.  The  ob- 
jectives prerequisite  to  trainer  exercises 
are  still  more  elemental,  and  they  constitute 
the  content  core  of  ground  school  instruction. 
Typically,  the  student  would  go  through 
several  evolutions  of  the  cycle  between  ground 
school  and  trainer  exercises  before  cycling  up 
to  an  aircraft  flight,  and  the  objectives  to 
be  acconplished  in  the  first  aircraft  flight 
would  be  less  difficult  than  those  scheduled 
for  later  flights.  At  each  stage  in  the 
syllabus,  the  student  is  prepared  to  advance 
to  the  next  stage. 

Segment -by-segment,  the  ISD  team  organizes  re- 
lated behavioral  objectives  into  lessons. 
Elemental  lessons  are  placed  ahead  of  more 
difficult  ones  In  the  syllabus,  and  a test  is 
scheduled  for  each  lesson.  The  lessons  that 
pertain  to  a given  subject  matter  are  organ- 
ized into  instructional  units,  and  an  equip- 
ment exercise  is  scheduled  at  the  end  of  each 
unit.  Finally,  the  various  units  of  instruc- 
tion are  structured  Into  a framework  that 
forms  the  course  syllabus. 

Not  only  do  course  syllabi  serve  as  maps  of 
the  Instructional  sequences  for  the  entire 
training  program,  but  also  they  provide  the 
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ISO  team  with  a sufficiently  complete  picture 
of  the  program  to  permit  an  accurate  analysis 
of  the  new  program's  training  support  require- 
ments. Consequently,  It  Is  at  tnls  stage  that 
the  ISO  team  estimates  the  total  personnel, 
equipment,  services,  materials,  and  facilities 
that  will  be  required  to  complete  the  develop- 
ment of  the  training  program,  and  to  Implement 
and  maintain  It  throughout  Its  life  cycle. 

For  example,  the  trainer  specifications  gener- 
ated earlier  during  task  selection  are  now  re- 
examined to  determine  If  each  trainer  will  en- 
compass a sufficient  number  of  objectives  to 
be  utilized  fully,  and  If  each  Is  the  least 
expensive  device  that  can  be  effective  In 
training  the  specified  objectives.  Similarly, 
the  classroom  media  selected  earlier  are  now 
evaluated  to  determine  which  media  are  most 
feasible  for  presenting  the  Information  within 
each  lesson.  The  answers  to  these  questions 
enable  the  ISD  team  to  estimate  the  purchase, 
modification,  production,  and  operating  costs 
required  for  the  various  trainers  and  Instruc- 
tional media.  However,  In  order  to  estimate 
personnel,  services,  and  facilities  require- 
ments, the  ISO  team  must  construct  a dally 
time-based  class  schedule  for  all  courses  In 
the  training  program.  From  this,  the  team  can 
establish  the  student  flow  throughout  the  pro- 
gram and  the  support  requirements  on  a lesson- 
by-lesson  basis. 

Training  support  requirements  analysis  pro- 
vides the  Information  needed  by  planners  and 
managers  to  assure  the  availability  of  criti- 
cal support  resources  needed  for  satisfactory 
completion  of  the  training  program  and  Its 
ultimate  Implementation  and  maintenance.  By 
basing  this  analysis  on  completed  course 
syllabi,  a major  source  of  potential  program 
failure  is  avoided,  l.e.,  failure  due  to  the 
unavailability  of  some  key  Instructional  re- 
source around  which  much  of  the  training  pro- 
gram implementation  Is  dependent.  Rigorous 
support  requirements  planning  at  this  stage 
in  the  ISD  process  ensures  that  only  those 
resources  available  to  the  program  are  In- 
corporated Into  Its  design. 

Now  that  the  skeleton  of  the  program  has  been 
built  and  the  resources  needed  to  support  It 
have  been  established,  the  time  has  come  for 
the  ISD  team  to  put  some  meat  on  the  bones, 
l.e.,  to  write  the  lesson  specifications.  The 
critical  subject  content  and  teaching  strategy 
must  now  be  specified  for  each  behavioral  ob- 
jective In  the  course  syllabus.  For  example, 
If  In  order  to  achieve  a particular  objective, 
the  student  must  learn  a rule,  a definition, 
a fact,  or  a procedure,  the  ISD  team  must 
state  It  explicitly.  Likewise,  If  a particu- 
lar kind  of  explanation,  or  mnemonic  aid,  will 
facilitate  learning  of  the  objective,  this 
also  Is  specified.  In  addition,  the  ISD  team 
determines  the  various  kinds  of  Illustrative 
examples,  practice  problems,  and  test  Items 


that  will  best  exemplify  and  evaluate  the  be- 
havior called  for  In  the  objectives.  The 
team  also  stipulates  the  graphic  Illustrations 
to  be  Included  In  the  Instructional  material 
for  each  objective.  After  this  has  been  com- 
pleted for  all  the  objectives  covered  In  a 
given  lesson,  a lesson  format  guide  Is  pre- 
pared which  explains  how  the  material  In  each 
segment  will  be  organized,  and  how  the  In- 
dividual segments  will  be  tied  together  to 
form  the  lesson.  The  output  from  this  stage 
of  the  ISD  process  Is  a set  of  tight  guide- 
lines that  will  be  used  to  control  the  organi- 
zation of  the  detailed  subject  matter  content. 

The  guidelines  help  to  avoid  the  kind  of  In- 
struction which  gives  either  too  much,  or  too 
little  attention  to  certain  materials,  misses 
the  point,  or  buries  It  In  a mass  of  detail, 
or  Ignores  fundamental  considerations  of 
teaching  strategy.  Proper  lesson  specifica- 
tion ensures  that  the  principles  to  be 
learned  In  any  group  of  behavioral  objectives 
determine  both  the  kind  and  degree  of  detail 
given,  as  well  as  the  strategy  that  Is  chosen 
to  teach  It. 

The  lesson  specifications  and  format  guides 
provide  the  ground-work  for  actual  lesson 
authoring.  Working  from  the  specifications  of 
lesson  contents  and  format,  the  ISD  team 
writes  out  paper  and  pencil  versions  of  the 
final  instructional  materials.  After  review 
and  editing,  these  materials  go  Into  prototype 
production  for  use  In  small-scale  tryouts  with 
real  students.  Usually,  these  prototype  1n- 
structlonal  materials  are  scheduled  for  try- 
outs as  they  are  being  developed.  By  means 
of  these  tryouts,  the  ISD  team  Is  able  to 
determine  whether  the  materials  are  actually 
effective  In  bringing  about  the  desired 
learning,  and  to  establish  whether  the  mate- 
rials are  palatable  to  the  students.  Instruc- 
tional materials  that  are  found  to  be  weak  In 
either  respect  are  revised  and,  if  necessary, 
tried  out  again. 


This  procedure  maximizes  the  probability  that 
the  instructional  materials  will  be  success- 
ful once  the  training  program  Is  implemented. 
After  the  materials  have  undergone  their  final 
production,  the  finished  product  Is  a package 
that  comes  as  close  to  guaranteeing  effective 
Instruction  as  can  be  provided  by  modern  tech- 
nology. However,  In  order  for  this  training 
package  to  remain  effective  throughout  Its 
life  cycle.  It  will  not  only  nave  to  be 
managed  properly,  but  It  will  need  to  be 
amenable  to  continuous  evaluation  and  up- 
dating. 

Once  the  Instructional  media  and  materials 
have  entered  the  final  production  phase,  the 
ISD  team  can  turn  Its  attention  to  developing 
a plan  that  will  control  Implementation, 


for  assuring  that  the  needed  modifications 
will  be  made  In  a timely  and  smooth  manner. 


operation,  and  long-term  evaluation  of  the 
training  program.  It  is  at  this  point  that 
the  basic  Instructional  management  system  Is 
established  this  system  defines  the  role  of 
Instructional  personnel,  the  student  manage- 
ment procedures,  and  the  procedures  for  re- 
source allocation  and  scheduling. 

Built  into  the  management  process  Is  a quality 
control  system  designed  to  continuously  assess 
the  effectiveness  and  palatablllty  of  the  In- 
structional materials,  as  well  as  the  Instruc- 
tional management  system  Itself.  The  objec- 
tive of  this  quality  control  system  Is  to  pro- 
vide a mechanism  for  identifying  those  pieces 
of  Instruction  that  require  modification,  and 


This  Is  the  final  task  of  the  ISO  team.  Upon 
its  completion,  the  team  will  have  developed, 
designed,  and  produced  a superior  Instruction' 
al  system,  one  which  can  be  Implemented  and 
managed  efficiently  for  the  lifetime  of  the 
program.  The  built-in  quality  control  system 
assures  that  the  program  will  always  accom- 
plish what  It  was  designed  to  do  and  that  Its 
materials  will  be  revised  to  reflect  the 
changes  in  conditions  and  constraints  that 
will  probably  lie  in  the  future.  Such  a pro- 
gram should  be  as  fresh  and  effective  at  the 
end  of  Its  life  cycle  as  it  was  at  the 
beginning. 


ABOUT  THE  AUTHORS 


CD R JOSEPH  F.  FUNARO  -it,  Deputy  fo> i Training  System  Development  cut 
the  Naval  Training  Equipment  Center.  Hit  fields  of  expertise  are 
human  factors  engineering  and  instructional  system  development. 

He  directed  the  RSD  program  uthlch  developed  the  Naval  Air  System 
Command/Naval  Training  Equipment  Center  model  described  In  the  present 
paper.  He  is  currently  directing  the  application  of  the  model  to 
numerous  aviation  ISO  program  Including  the  SH-2,  E-2,  F-4,  F-14 
and  LAMPS  MK  III.  CD R Funaro  holds  B.S.  and  M.A.  degrees  from  Ohio 
State  University  and  the  Ph.D.  In  engineering  psychology  from  Rice 
University. 


DR.  8.  E.  MULLIGAN  is  Associate  Professor  Biological  and 
Experimental  Psychology  at  the  University  of  Georgia,  Athens,  Georgia. 
Research  specialties  are  In  acoustic  sensory  system  and  communication, 
and  applied  specialties  are  In  human  factors  engineering  and  training. 
He  is  a consultant  for  Telcom  System,  Inc.,  Arlington,  Virginia; 
for  the  Naval  Air  System  Command;  and  for  the  Naval  Training  Equipment 
Center. 


19/20 


TRAINING  ANALYSIS  FROM  AN  OPERATOR'S  POINT  OF  VIEW 

LARRY  H.  NOWELL 
Logicon,  Inc. 


It  is  difficult  to  single  out  a group  of 
people  and  say  that  their  job  performance 
affects  the  combat  readiness  of  the  fleet 
without  mentioning  the  Operations  Specialists 
in  Combat  Information  Center  (CIC).  The 
operators  collect,  translate,  display,  evalu- 
ate, disseminate  data  and  make  recommenda- 
tions in  relation  to  Anti-air  Warfare 
Surface  Warfare,  Subsurface  Warfare, 
Electronic  Warfare,  gunfire  support,  ship 
maneuvers,  data-link  operations  and  communi- 
cations. I am  not  trying  to  say  that  they 
alone  do  all  of  these  tasks,  but  it  is 
normally  their  input  that  affects  the  total 
outcome.  The  Tactical  Action  Office  (TAO) 
cannot  do  their  job  unless  targets  are  de- 
tected and  displayed.  Surface  and  Sub- 
surface Warfare  would  wind  up  in  knuckles 
(disturbance  in  the  water  caused  by  ships 
making  tight  turns  that  can  be  mistaken  for 
submarines)  without  maneuvering  boards  and 
NC-2  plotters.  The  Air  Intercept  Controllers 
(AIC)  and  Antisubmarine  Air  Controllers 
(ASAC)  are  the  data  and  voice  links  to  all 
aircraft.  Many  Officers  of  the  Deck  (OODs) 
have  been  saved  from  a near  miss  or  going 
aground  by  alert  operators  in  CIC. 

I am  proud  of  the  fact  that  I was  an 
operator  for  20  years  in  the  Navy.  The  first 
school  I attended  in  the  Navy  seemed  to  set  a 
pattern  for  things  to  come.  I spent  more 
than  two  years  attending  schools  and  eight 
years  teaching  operator  training.  At  least 
half  of  my  Naval  career  was  spent  in  schools. 
I have  felt  the  enlightenment  of  learning, 
the  satisfaction  of  teaching,  and  the 
frustration  of  failure.  Failure  to  teach 
someone  a skill  you  know  he  is  capable  of 
learning  can  be  very  frustrating. 

I experienced  many  benefits  as  a result  of 
the  time  I spent  in  school.  In  1972  I was 
awarded  the  Distinguished  Service  Medal.  I 
am  the  only  enlisted  man  to  ever  receive  this 
nation's  third  highest  award  for  combat 
action. 

After  receiving  so  much  from  Navy  schools, 
I feel  l am  extremely  lucky  to  be  able  to 
continue  working  on  training  systems  in  the 
civilian  coimunity. 

In  this  paper  I have  tried  to  describe 
some  problems  in  the  training  of  operators, 
the  reasons  why  I feel  they  have  occurred, 
and  my  recommended  solution.  The  main 
problems  in  operator  training  (as  I see  them) 
are  the  following: 


a.  training  time  is  limited 

b.  present  teaching  methods  are  out  of 
date 

c.  little  standardization  in  NTDS  programs 

d.  shortage  of  qualified  instructors 
aboard  ship 

e.  reduction  in  training  time  for  quali- 
fication of  new  personnel  reporting 
aboard  ship 

f.  high  cost  and  increasing  number  of 
operational  consoles. 

Training  in  the  Navy  is  a way  of  life. 

The  Navy  and  Naval  personnel  are  very  proud 
of  their  schools.  The  Navy  schools  are  one 
of  the  reasons  I decided  to  join  the  Navy. 

The  schools  I attended  allowed  me  to  perform 
my  job  and  take  pride  in  my  performance. 

Being  able  to  obtain  job  satisfaction  was  a 
motivating  factor  to  stay  in  the  service. 

Rate  training  is  taught  primarily  in 
schools.  The  formal  training  being  performed 
concentrates  on  teaching  basic  knowledge  that 
cannot  be  picked  up  easily  aboard  ship.  The 
instructors  assigned  to  the  schools  are 
recommended  for  instructor  training  by  their 
previous  commands  before  being  assigned  to 
teach  at  a service  school.  Each  instructor 
attends  an  Instructor  Training  School  to 
develop  the  needed  skills  of  presenting 
material  to  the  students.  Much  effort  is 
expended  in  preparing  sailors  for  instructor 
duty.  The  primary  restricting  factor  to  the 
school  is  time.  Time  spent  at  school  is  time 
away  from  the  ship. 

Time  was  not  always  a problem.  Training 
on  the  job  was  always  the  primary  teaching 
site.  Each  petty  officer  is  expected  to 
conduct  on-the-job  training.  A portion  of 
each  watch  is  spent  in  training.  The  best 
way  to  pass  a mid-watch  was  to  have  a well - 
planned  training  exercise.  The  bridge  would 
always  welcome  training  during  slow  watches. 
Ships  steaming  in  company  would  join  in  the 
training  exercises.  Predeployment  training 
plans  determine  where  the  main  thrust  of  the 
training  would  be  placed  during  a deployment. 
New  operators  are  properly  introduced  to  the 
job  they  are  expected  to  do  in  a one-on-one 
(instructor-to-student)  environment.  An 
operator  spends  many  hours  under  close 
supervision  before  he  is  allowed  to  operate 
on  his  own. 

For  a long  time  I felt  we  had  a good 
balance  in  training.  The  schools  were 
teaching  the  needed  information,  and  everyone 
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aboard  ship  was  doing  his  part  to  ensure  good 
performance  by  the  operators. 

As  a result  of  the  reduced  size  of  the 
fleet,  an  acute  shortage  of  experienced 
personnel  aboard  ship  and  the  time  to  train 
new  men  reporting  on  board  forced  the  fleet 
to  require  better  trained  personnel.  The 
mark  was  set  at  requiring  (with  a minimum 
indoctrination  period)  the  graduates  to  be 
condition  III  qualified. 

Condition  III  is  wartime  steaming.  To 
expect  a graduate  to  step  aboard  ship  and, 
after  a couple  of  watches,  be  qualified  to 
"fight  the  ship"  leaves  everyone  with  a 
problem.  I feel  the  fleet  is  willing  to  give 
the  schools  some  additional  time  for 
training,  but  not  a great  deal. 

The  Naval  Tactical  Data  System  (NTDS)  has 
to  receive  at  least  some  of  the  credit  for 
the  training  problems.  Only  three  ships 
carried  NTDS  for  a long  time.  Most  reports 
from  the  NTDS  ships  were  good.  Everyone 
wanted  to  get  aboard  one,  but  only  a few  were 
allowed.  To  correct  some  initial  problems 
and  keep  the  NTDS  programs  running,  changes 
were  being  made  faster  than  the  school  could 
keep  up  with,  resulting  in  the  output  of 
outdated  information.  As  we  were  about  to 
get  everything  together,  a new  display  con- 
sole hit  the  street.  Most  people  liked  the 
new  console  better.  The  school  now  had  a 
problem.  The  school  was  teaching  the  equip- 
ment. This  knob  does  this  when  you  throw  it, 
see.  The  school  even  threw  in  some  informa- 
tion about  the  computers  but  somehow  never 
got  around  to  teaching  the  operators  their 
job.  It  could  have  been  the  way  operator 
training  has  always  been  taught,  or  that  the 
instructors  (most  instructors  had  never  been 
aboard  an  NTDS  ship)  did  not  know  the  jobs. 
Primarily,  the  schools  started  teaching  the 
new  consoles  and  the  differences  between  the 
consoles.  Other  events  occurred  to  add  to 
the  problem  such  as: 

a.  The  programmer  responsibility  was 
changed  from  Navy  officers  to 
civilians,  creating  a new  learning 
period  for  the  civilians. 

b.  Four  consoles  are  being  used  in  the 
fleet  today  and  more  are  being 
developed.  To  operate  a piece  of 
equipment  effectively,  personnel  must 
become  intimately  involved  with  the 
controls.  Since  the  equipment  becomes 
a part  of  the  operator  (an  extension  of 
his  need  for  information),  he  must  be 
able  to  extract  the  needed  information 
without  looking  at  the  controls  or 
breaking  his  line  of  concentration. 

Each  new  console  has  rearranged  the 
operator  controls,  which  means  more 
time  and  money  spent  in  retraining. 


c.  Two  different  program  centers  are 

writing  programs  with  little  standardi- 
zation. Four  major  program  develop- 
ments have  occurred,  each  with  some 
modification,  with  each  program  chang- 
ing the  operator  controls.  For  an 
example,  on  the  east  coast  the  opera- 
tors are  taught  the  Utility  Mode  which 
expands  the  operator's  capability  to 
perform  needed  functions.  All  of  the 
smaller  ships  on  both  the  east  coast 
and  west  coast  have  the  Utility  Model. 
The  west  coast  teaches  Split  Labels 
which  does  basically  the  same  thing, 
but  there  is  no  changing  of  modes.  All 
the  large  ships,  east  and  west  coast, 
use  ^hem.  It  has  been  planned  to  teach 
all  operators  on  one  coast.  Wonder 
which  program  they  will  teach?  There 
is  an  estimated  requirement  to  teach 
3,000  students  a year.  Starting  a class 
each  week  which  lasts  for  3 weeks  (very 
minimum  training  time)  means  that  we 
have  to  contend  with  60  students  per 
class  x 3 weeks  = 180  students  in 
training  every  day.  Each  student  needs 
a minimum  of  4 hours  on  a console  each 
day,  which  equals  720  equipment  hours. 
Allowing  for  16  hours  a day,  45  con- 
soles are  needed,  up  and  running  all 
the  time,  every  day. 

To  teach  the  proper  program,  we  could 
split  the  class  into  those  going  to  the  east 
coast  and  those  going  to  the  west  coast. 

Then,  divide  each  of  these  groups  into  four 
or  five  units  to  allow  for  console  differ- 
ences. We  need  8 to  10  mockups  totally  dedi- 
cated for  operator  training.  Each  mockup 
would  require  at  least  6 or  8 consoles  to 
allow  for  a difference  in  student  load.  A 
total  of  80  consoles  would  be  required  allow- 
ing for  downtime  and  growth.  My  estimate, 
based  on  a recent  study,  indicates  that  the 
cost  of  using  shipboard  equipment  to  train 
operators  is  approximately  $1  million  per 
station,  or  $80  million.  Of  course  the  cost 
may  go  down  a bit  if  purchases  are  made  in 
quantity. 

I have  confidence  in  the  Naval  school 
system,  and  since  the  two  programming  centers 
have  shown  signs  of  coordinating  their 
efforts  our  next  program  development  effort 
should  be  one  of  the  best.  Instructional 
System  Development  (ISO)  and  the  systems 
approach  to  training  are  the  best  things  to 
come  along  to  help  operator  training.  As  ISD 
is  incorporated  into  the  schools  and  the 
instructors  become  more  familiar  with  it,  I 
feel  our  teaching  methods  will  work 
themselves  out. 

Given  that  this  is  true  and  the  instruc- 
tors have  improved  their  teaching  techniques, 
the  remaining  problem  is  the  number  of  con- 
soles in  use  in  the  fleet.  One  solution 
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might  be  to  pack  the  schools  with  all  types 
of  consoles  found  in  the  fleet  and  teach  the 
trainee  on  the  one  he  has  aboard  his  ship. 
Even  if  money  and  space  were  available,  this 
shotgun  technique  is  bound  to  fail.  It  is 
now  time  to  analyze  the  problem  in  a logical 
manner. 


student/training  system  Interface  is  needed. 
An  off-the-shelf  CRT  and  keyboard  will  do 
nicely.  Introductory  information,  student 
sign-on/off,  new  material,  some  teaching  and 
testing  of  knowledge  subjects,  review  of  past 
performance,  and  student  status  will  be 
conducted. 


We  need  a training  program  to  bring  the 
student  up  to  a defined  level  of  proficiency. 
To  do  this  we  need  to  measure  the  student's 
ability  to  perform  required  tasks.  We  need  a 
program  that  is  objective  and  adaptive  to  the 
student's  needs. 

The  use  of  operational  programs  seriously 
limits  the  teaching  capabilities  of  any 
training  system.  The  basic  requirements 
needed  to  teach  the  operators  their  perform- 
ance skills  have  not  changed;  they  are  still 
the  same  as  when  NTDS  was  first  introduced. 
Some  additional  requirements  have  been  added, 
but  the  basic  needs  of  the  system  that  re- 
quire manual  input  and  modification  have  not 
changed.  The  basic  requirements  for  the 
operator  to  perform  his  task  are  needed  - 
nothing  else.  Teaching  operator  techniques 
establishes  a need  for  the  operator  to  obtain 
information  from  the  console  or  to  give  in- 
puts to  the  system.  These  basic  tasks  have 
not  changed  and  could  be  any  or  all  of  the 
fol lowing: 

a.  entering  new  tracks 

b.  tracking  targets 

c.  obtaining  bearing  and  range  to  a target. 

A handful  of  tasks  would  allow  most 
operators  to  perform  their  job  in  a condition 
III  environment.  There  is  no  NTDS  equipment 
anywhere  dedicated  to  individual  NTDS  opera- 
tor training.  That  bears  repeating!  There 
is  no  NTDS  equipment  anywhere  dedicated  to 
individual  NTDS  operator  training!!  Even  if 
four  mockups  were  available  with  four  differ- 
ent types  of  equipment,  the  training  center 
schools  would  be  fighting  for  their  use.  The 
NTDS  operator  schools  are  lucky  to  get  one 
mockup  for  training. 

To  divert  the  soaring  cost  of  shipboard 
equipment  and  to  obtain  a usable  training 
console  that  would  allow  for  a learning 
environment,  I propose  an  adaptive  training 
console.  An  adaptive  training  console  would 
not  only  adapt  to  the  needs  of  the  student 
but  also  to  hardware  and  software  changes  a* 
a minimum  cost.  A console  designed  just  for 
teaching!  Let's  take  a look  at  it.  First,  a 


Using  a preparatory  station  is  very  cost- 
effective.  While  the  primary  station  is 
being  used  by  a prepared  student,  a smaller, 
cheaper  station  is  preparing  the  next  stu- 
dent. The  primary  training  station  will  also 
have  an  interface,  CRT,  keyboard,  and  the 
Adaptive  Training  Console  (ATC).  The  ATC 
presents  and  configures  itself  according  to 
the  level  of  the  student. 

After  identifying  the  basic  jobs  of  the 
operator,  the  training  system  presents  the 
job  and  the  need  for  information  from  the 
console  or  a need  to  input  data.  To  this 
point  the  console  is  blank.  No  knobs, 
switches,  alerts,  controls . or  alarms  (see 
Figure  1).  As  the  need  for  a switch,  knob, 
alert,  control  , or  alarm  arises,  it  will  be 
presented  by  the  training  system  (see  Figure 
2). 


As  the  student's  knowledge  and  skill 
builds,  so  does  the  console.  This  training 
system  is  designed  to  give  prompts  to  the 
student  when  needed.  It  is  easily  adapted 
to  other  equipment  design  for  training,  data 
readouts,  and  IFF  control  boxes.  A line 
printer  will  be  furnished  for  hard  copy 
prirttouts. 

After  the  basic  skills  are  learned,  the 
console  adapts  to  the  required  operational 
console.  The  ATC  adapts  to  the  individual 
needs  of  the  student  by  presenting  the  like- 
ness of  the  console  aboard  his  ship  to  the 
student.  Each  ATC  has  the  capability  to  du- 
plicate the  functions  of  each  console  being 
used  in  the  fleet  (see  Figure  3). 

The  adaptive  training  console  can  be  re- 
programned  by  the  training  center  to  adapt  to 
hardware  and  software  changes  as  long  as  the 
basic  functions  do  not  change.  A change  in 
operator  functions  would  require  minor 
software  modification. 

The  adaptive  training  console  will  give  the 
schools  a base  from  which  to  establish  the 
most  effective  operator  training  available. 

The  adaptive  training  console  may  very  well  be 
the  last  console  the  school  will  buy!! 
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USER  ACCEPTANCE  IN  AN  AUTOMATED  SPEECH 
TECHNOLOGY  BASED  TRAINING  SYSTEM 

MARY  HICKLIN  AND  GAIL  SLEMON 
Logicon,  Incorporated 


INTRODUCTION 

Computers  are  no  longer  deaf  and  dumb! 

The  automated  speech  technologies  (AST)  have 
made  it  possible  for  computers  to  converse 
with  their  users.  Now  computers  can  actually 
understand  the  spoken  word  and  can  speak  as 
easily  as  they  can  print.  In  theory,  simula- 
tion of  the  verbal  behavior  of  persons  in  the 
environment  is  now  possible.  In  practice, 
the  current  state  of  the  art  imposes  some 
rather  severe  limitations  on  such  simulations. 
This  paper  describes  some  practical  problems 
encountered  in  the  development  of  a speed) 
technology  based  training  system  for  use  in  an 
environment  where  user  acceptance  is  of  para- 
mount importance.  The  design  solutions  to 
these  problems  suggest  that  the  state  of  the 
art  has  advanced  to  the  point  where  speech 
technologies  can  be  integrated  successfully 
into  today's  operational  simulators,  albeit 
with  care  in  demanding  environments. 

Background.  Logicon 's  Advanced  Systems 
Department  has  "long  been  concerned  with  the 
automation  of  training.  Since  1969,  we  have 
been  studying  the  application  of  the  auto- 
mated speech  technologies  to  the  training  en- 
vironment. In  1974  we  delivered  a laboratory 
version  of  the  Ground  Controlled  Approach  Con- 
troller Training  System  (GCA-CTS)  to  the  Naval 
Training  Equipment  Center  (NTEC)  in  Orlando, 
Florida  to  demonstrate  the  feasibility  of  a 
speech  recognition  based  training  system. 

Since  that  time,  we  have  incorporated  the 
speech  technologies  in  other  systems  including 
the  Automated  Flight  Training  System  designed 
for  the  Air  Force  and  an  Automated  Command 
Response  Verification  System,  a maritime 
safety  system  studied  for  the  Department  of 
Transportation.  We  have  also  continued  to  re- 
fine the  original  GCA-CTS  in  accordance  with 
the  results  of  experiments  conducted  by  the 
Scientific  Officer,  Dr.  Robert  Breaux.  We  are 
presently  building  an  experimental  prototype 
version  of  the  GCA-CTS  for  evaluation  at  the 
Naval  Air  Technical  Training  Center  in  Memphis, 
Tennessee.  The  remainder  of  this  discussion 
will  focus  on  this  GCA-CTS  work,  which  poses 
the  most  challenging  user  acceptance  problems 
encountered  to  date. 

The  GCA-CTS.  The  GCA-CTS  will  employ 
coimercially  available  hardware  including  two 
Data  General  Eclipse  S/130  minicomputers,  a 
Threshold  Technology  Threshold  500  speech  pre- 
processor, a Federal  Screw  Works  VS  6.4  Speech 


Synthesizer,  and  a Megatek  MG552  Graphic  Dis- 
play Processor.  The  GCA-CTS  will  be  designed 
to  train  students  to  interpret  precision  ap- 
proach radar  information  and  to  give  the  well- 
defined  verbal  advisories  which  will  enable 
the  pilot  to  make  a safe  approach  even  in  con- 
ditions of  low  visibility  and  without  NAVAID 
receiving  equipment  in  the  aircraft. 

The  problem  of  training  GCA  controllers 
was  originally  chosen  for  study  for  several 
reasons.  First  of  all,  NTEC  was  interested 
in  applying  the  concepts  of  automated  adaptive 
training  to  air  controller  training.  Until 
the  advent  of  automated  speech  recognition 
however,  machine-measurement  of  verbal  task 
performance  was  not  feasible.  Automated 
speech  recognition  now  makes  it  possible  for 
the  computer  to  determine  the  accuracy  of  what 
was  spoken  and  grade  student  performance  ac- 
cordingly. With  detailed  performance  assess- 
ment, it  can  automatically  structure  the 
training  course  to  conform  to  the  needs  of  the 
individual  student.  Secondly,  present  train- 
ing for  one  student  requires  the  attention  of 
both  an  instructor  and  a pseudo  pilot  who  con- 
trols a simulated  radar  target.  This  expen- 
sive resource  allocation  made  the  task  a 
likely  candidate  for  automation.  Finally, 
since  the  verbal  advisories  are  standardized 
and  precise,  the  requirements  seemed  to  be 
within  the  scope  of  the  isolated  word  recogni- 
tion art.  Commercially  available  speech  rec- 
ognition systems  are  capable  of  recognizing 
a limited  vocabulary  of  words  or  phrases, 
spoken  in  isolation,  using  reference  patterns 
collected  from  the  individual  talker.  A GCA 
controller  training  system  therefore  provided 
an  ideal  test  bed  for  the  concept  of  applying 
the  AST  to  the  training  environment. 

The  system  which  is  currently  being  de- 
signed will  serve  as  an  experimental  task 
trainer  for  use  by  U.S.  Navy  enlisted  men  and 
women.  It  will  provide  relatively  standard 
simulation  of  the  radar  environment  and  air- 
craft dynamics.  No  pseudo  pilot  will  be  need- 
ed because  it  will  employ  a pilot  simulation 
whose  ears  will  be  the  speech  recognition 
capability  and  whose  voice  will  be  the  speech 
synthesizer.  Furthermore,  the  routine  duties 
of  the  instructor  will  be  simulated  so  that 
the  GCA-CTS  can  provide  instruction,  present 
problems  based  upon  the  trainee's  level  of 
skill,  supply  performance  assessments,  and 
even  administer  and  score  a final  examination. 
Simulation  of  these  routine  instructional 
duties  will  free  the  instructor  to  fill  the 
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more  demanding  role  of  training  manager  and 
will  provide  him  with  the  data  to  do  this  more 
effectively. 

USER  ACCEPTANCE  CONSIDERATIONS 

The  GCA-CTS  is  an  ambitious  project  both 
in  terms  of  the  scope  of  the  required  simula- 
tion and  because  of  its  reliance  on  state  of 
the  art  technologies.  The  successful  labora- 
tory demonstrations  have  focused  upon  the 
technical  risk  areas  and  have  revealed  poten- 
tial risks  in  the  area  of  user  acceptance. 

If  the  system  proves  difficult  to  use,  irri- 
tating to  listen  to,  or  fails  consistently  to 
recognize  the  spoken  advisories,  the  trainee's 
frustration  will  probably  impede  learning. 

The  following  paragraphs  describe  many 
potential  AST  related  user  acceptance  problems 
and  present  the  diverse  solutions  which  will 
be  employed  in  the  GCA-CTS.  These  include: 

a.  teaching  the  student  to  use  the 
system  properly 

b.  design  of  a training  course  which 
complements  AST  requirements 

c.  modification  of  state  of  the  art 
speech  recognition  algorithms  to 
accommodate  the  GCA  vocabulary 

d.  addition  of  speech  understanding 
logic  to  augment  speech  recognition 

e.  provision  of  effective  feedback 

f.  use  of  discretion  in  the  application 
of  speech  generation. 

The  very  diversity  which  makes  the  topics  dif- 
ficult to  weave  into  a logical  progression 
illustrates  the  point  that  a concern  for  user 
acceptance  must  pervade  all  aspect  of  system 
design. 

Training  GCA-CTS  Users.  Proper  use  of 
the  training  system  will  be  the  first  topic 
addressed  in  the  training  program.  The  rules 
of  microphone  placement  and  speech  level  pro- 
duction are  critical  to  good  speech  recogni- 
tion, but  are  easy  to  learn  and  employ.  There 
is  also  a less  obvious  component  of  proper 
system  use  which  has  been  labeled  "learning 
to  talk  to  the  system."  Introspection,  if  it 
may  be  admitted,  suggests  that  the  components 
of  this  art  include  confidence,  naturalness 
of  speech  and  consistency.  The  skill  is  easi- 
ly acquired,  yet  time  for  acclimatization  is 
very  important  to  good  speech  recognition. 
During  this  period  the  trainee  will  be  en- 
couraged to  experiment  with  the  system  to 
learn  that  it  really  can  recognize  what  he 
says,  and  to  discover  the  limitations  inherent 
in  automatic  speech  recognition.  With  this 
background,  he  will  be  ready  to  use  the  sys- 
tem effectively. 


A Training  Course  Which  Complements  AST. 
There  are  many  possible  approaches  to  teach- 
ing  GCA  controller  skills.  The  present 
course  requires  that  the  trainee  master  the 
entire  vocabulary  and  try  to  put  it  to  use  the 
first  time  he  plays  the  role  of  final  con- 
troller. At  first  the  instructor  stands  by 
and  prompts  him,  and  the  pseudo  pilot  (a 
trainee  also)  can  easily  understand  the  ad- 
visories despite  stylization  problems  and 
even  word  substitution  errors.  During  the  5 
days  of  training,  errors  decrease  dramati- 
cally and  the  trainee  emerges  as  a qualified 
controller. 

This  training  philosophy  is  not  suitable 
for  an  automated  training  system  however. 

The  automated  system  has  certain  strengths 
and  certain  limitations  when  compared  to  the 
human  instructor.  The  automated  system  can- 
not match  the  verbalization  error  tolerance 
of  the  human  listener  for  example,  but  it  can 
be  much  more  attentive  than  an  instructor  who 
is  responsible  for  several  students.  It  has 
the  flexibility  to  simplify  the  environment 
and  even  stop  the  approach  if  necessary  to 
illustrate  its  points.  It  can  also  contrive 
practice  approaches  which  require  use  of  only 
that  material  learned  to  date. 

To  take  advantage  of  the  tremendous 
power  of  the  automated  training  system  and  to 
minimize  the  impact  of  its  shortcomings,  a 
training  course  has  been  designed  by  our 
training  technologists  to  maximize  the  train- 
ing effectiveness  of  the  GCA-CTS.  This  train- 
ing will  be  in  accordance  with  the  principles 
of  errorless  learning  so  the  trainee  should 
never  learn  to  make  mistakes.  Furthermore, 
this  training  strategy  will  actually  take 
advantage  of  what  is  often  considered  to  be  a 
drawback  in  state  of  the  art  speaker  dependent 
speech  recognition,  namely  the  requirement  to 
configure  the  system  for  the  individual's 
speech  patterns.  Briefly,  the  training  strat- 
egy involves  dividing  the  GCA  controller  task 
into  its  component  parts  so  that  one  topic 
can  be  presented  at  a time.  The  simulated 
environment  will  be  manipulated  to  provide 
illustrative  examples  while  the  instructor 
simulation  provides  prompts.  The  trainee's 
speech  patterns  will  be  collected  at  this  time 
and  used  to  create  the  reference  patterns  for 
speech  recognition.  Thus  as  the  trainee 
learns  the  procedures  and  phraseology,  the 
system  will  be  learning  to  recognize  his  voice. 
After  this  phase,  the  system  will  present 
tasks  for  the  trainee  to  perform  without 
prompts.  These  will  be  scored.  Their  purpose 
will  be  to  allow  the  student  to  practice  the 
new  material  and  integrate  it  with  the  old. 

This  training  strategy  is  expected  to 
impact  user  acceptance  in  several  ways.  First, 
use  of  the  system  should  prove  rewarding. 
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Initial  speech  recognition  problems  will  be 
minimized  by  the  small  vocabulary  and  thor- 
ough training  so  the  goals  set  forth  should 
be  readily  attainable.  This  initial  success 
is  expected  to  bolster  the  trainee's  confi- 
dence in  the  system,  thus,  providing  an  imnor- 
tant  ingredient  for  future  recognition  success. 

Secondly,  the  formidable  task  of  config- 
uring the  system  to  recognize  the  large  GCA- 
CTS  vocabulary  has  been  integrated  with  task 
training  in  a way  that  will  be  transparent  to 
the  user.  User  acceptance  will  not  be  hind- 
ered by  an  enmasse  onslaught  of  phrase  repeti- 
tion requests.  In  addition,  the  strategy  em- 
ployed in  speech  pattern  collection  should 
ensure  representative  reference  patterns  which 
further  promote  speech  recognition  accuracy.- 

Finally,  the  automated  adaptive  system 
will  devote  all  of  its  resources  to  teaching 
the  individual  user.  It  will  ensure  that  each 
topic  is  mastered  in  a systematic  way  at  the 
trainee's  own  pace.  Any  needed  remediation 
will  be  provided  automatically.  Because  of 
this  highly  individualized  instruction,  the 
trainee  will  be  expected  to  attain  profici- 
ency in  all  of  the  GCA  control  tasks.  The 
proficiency  level  attainable  through  thorough, 
systematic,  task-orie.ited  learning  can  be  its 
own  reward  and  should  enhance  user  acceptance. 

Augmentation  of  Speech  Recognition  Algo- 
rithms^ In  addition  to  these  efforts  to  en- 
sure good  speech  recognition  by  training  the 
user  carefully,  the  traditional  speech  recog- 
nition algorithms  must  be  enhanced  to  accommo- 
date the  challenging  vocabulary.  The  long 
vocabulary  list  of  approximately  100  phrases 
contains  many  similar  items  and  consists  of 
both  very  long  and  very  short  utterances. 
Obviously  this  vocabulary  cannot  be  modified 
in  the  interests  of  good  speech  recognition 
because  it  is  defined  by  the  FAA  and  used  by 
all  GCA  controllers;  instead,  speech  recogni- 
tion will  have  to  be  modified. 

In  order  to  recognize  the  relatively 
large  vocabulary  in  real-time  and  with  high 
accuracy,  the  reference  patterns  will  be  par- 
titioned by  the  phase  of  the  approach,  the 
previous  transmission,  the  length  of  the 
phrase,  and  its  intended  destination.  Using 
all  these  clues,  the  branching  factor  can  be 
reduced  to  about  50  items  in  the  worst  case. 
This  is  similar  to  the  branching  factor  tested 
successfully  in  the  laboratory  GCA-CTS. 

To  distinguish  vocabulary  items  which 
vary  only  slightly,  a scheme  devised  by  Dr. 
Breaux  and  tested  in  the  laboratory  GCA-CTS 
will  be  employed.  The  technique  involves  com- 
paring similar  reference  patterns  to  find 
those  time  slots  which  are  significantly  dif- 
ferent. These  rows  are  then  correlated  with 


the  corresponding  rows  of  the  input  feature 
pattern.  The  procedure  effectively  causes 
the  pattern  recognition  algorithm  to  weight 
the  distinctive  portion  of  the  utterance  more 
heavily  than  the  similar  portions. 

The  GCA-CTS  vocabulary,  with  its  combi- 
nation of  long  and  short  utterances,  is  a 
problem  for  the  speech  recognition  algorithm, 
which  depends  on  a time  normalization  scheme 
to  extract  relevant  information  from  input 
utterances.  The  input  is  compressed  into  a 
pattern  of  standard  length  in  which  only  those 
features  which  are  reliably  present  are  set. 
This  compressed  array  is  compared  to  similarly 
constructed  reference  patterns.  The  optimum 
length  of  these  compressed  arrays  is  related 
to  the  length  of  the  input  utterance.  This 
becomes  problematical  when  phrases  as  diverse 
as  "eight"  and  "going  further  above  glidepath" 
must  be  recognized.  This  will  be  solved  by 
employing  two  sizes  of  reference  patterns 
based  on  the  length  of  the  utterance.  This 
may  even  prove  helpful  for  vocabulary  items 
of  medium  duration.  On  an  experimental  basis, 
both  reference  patterns  will  be  used  for  these 
items  and  the  results  will  impact  the  speech 
recognition  module's  measure  of  confidence 
in  its  decision. 

The  Requirement  for  Speech  Understanding. 
To  ensure  user  acceptance,  the  system  must 
not  make  the  mistakes  which  strict  reliance 
upon  speech  recognition  in  this  demanding  en- 
vironment could  produce.  There  are  sources 
of  information  in  the  simulation  environment 
as  well  as  a priori  information  which  can  be 
used  to  ensure  the  system  understands  what 
was  spoken. 

First  of  all,  the  speech  understanding 
module  will  atempt  to  distinguish  between 
controller  errors  and  recognition  errors,  and 
will  compensate  for  the  latter.  An  example 
will  help  clarify  this  point.  To  the  machine 
ear,  the  advisories  "well  above  glidepath" 
and  "well  below  glidepath"  sound  very  much 
alike.  Confusing  these  two  is  a gross  error 
which  the  trainee  is  not  likely  to  make.  If 
such  an  error  occurs,  the  system  will  assume 
the  trainee  has  given  the  correct  advisory. 
Trainee  errors,  on  the  other  hand,  are  more 
likely  to  involve  a confusion  between  contig- 
uous aircraft  positions  such  as  between  "above 
glidepath"  and  "slightly  above  glidepath." 
These  phrases  are  readily  distinguishable  to 
the  speech  recognition  algorithm,  so  the  sys- 
tem will  be  able  to  score  the  incorrect  ad- 
visory. 

The  speech  understanding  module  will  also 
compensate  for  typical  trainee  stylization 
errors  whenever  possible.  GCA-CTS  will  em- 
ploy isolated  phrase  recognition  and  there- 
fore there  must  be  slight  pauses  between  the 
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digits  given  in  turn  commands.  Because  of 
their  previous  training,  students  are  expected 
to  forget  this  important  requirement.  Audio 
tapes  reveal  that  a frequent  error  is  a fail- 
ure to  pause  before  the  first  digit  of  a 
heading.  Trainees  will  say  for  example,  "turn 
right  heading  one  (pause)  five  (pause)  five." 
This  utterance  will  probably  trigger  a low 
confidence  recognition  of  "turn  right  heading" 
followed  by  correct  recognition  of  two  digits. 
Since  the  first  digit  is  always  the  same, 
speech  understanding  will  assume  this  error 
occurred  if  two  reasonable  digits  follow  a 
heading  command  and  will  compensate  for  it. 

In  parallel  development  work,  we  are  investi- 
gating an  extremely  promising  limited  con- 
tinuous speech  recognition  algorithm  called 
LISTEN  (Logicon's  Initial  System  for  the 
Timely  Extraction  of  Numbers)  which  would  ob- 
viate the  need  for  stylization  of  heading 
coranands.  The  exciting  results  of  the  first 
phase  of  this  work  were  presented  at  the  De- 
cember, 1977  Conference  on  Voice  Technology 
for  Interactive  Real-Time  Command/Control 
Systems  Application  at  NASA,  Ames  Laboratory, 
Sunnyvale,  California.  The  technique  is  being 
tested  now  in  a laboratory  training  system, 
and  we  expect  it  will  soon  be  ready  for  in- 
corporation in  the  GCA-CTS  and  other  opera- 
tional trainers. 

Finally,  speech  understanding  will  moni- 
tor the  speech  level  to  detect  fluctuations 
significant  enough  to  affect  recognition  ac- 
curacy. It  will  inform  the  trainee  through 
the  simulated  pilot  when  transmissions  become 
weak. 

Effective  Feedback.  Despite  the  precau- 
tions taken  to  ensure  errorless  learning, 
consistently  error-free  performance  in  the 
complex  GCA  controller  environment  will  prob- 
ably remain  a lofty  ideal  which  can  only  be 
imperfectly  realized.  Therefore,  effective 
feedback  will  be  provided  whereby  the  user 
can  understand  and  learn  from  his  mistakes. 

In  this  unique  training  environment,  these 
mistakes  will  include  stylization  errors  which 
cause  recognition  failures.  Dr.  Breaux  added 
a replay  capability  to  the  laboratory  GCA-CTS 
and  it  was  found  to  be  an  effective  feedback 
technique  for  procedural  errors.  In  that  sys- 
tem the  speech  synthesizer  repeats  the  train- 
ee's advisories  and  gives  rule  explanations 
when  an  error  is  encountered.  He  noted,  how- 
ever, that  "message  not  understood"  reports 
proved  especially  frustrating  to  the  students. 
Many  times  a student  was  convinced  that  he 
had  uttered  the  correct  advisory  but  he  had  no 
way  to  argue  with  the  computer  or  to  under- 
stand why  the  recognition  failure  occurred. 

For  example,  he  would  remember  that  he  had 


said  "slightly  above  glidepath"  but  would 
not  realize  he  had  paused  in  mid-phrase, 
making  the  advisory  unintelligible  to  speech 
recognition. 

To  add  to  replay's  usefulness  as  a feed- 
back technique  and  to  reduce  the  frustra- 
tions associated  with  recognition  failures, 
a speech  input  digitizer  will  be  designed 
to  record  the  trainee's  advisories.  The 
replay  will  then  consist  of  an  actual  re- 
cording of  the  trainee's  speech,  synchro- 
nized with  the  visual  display.  From  this, 
the  student  should  be  able  to  understand 
the  cause  of  any  recognition  failures  which 
occur.  In  addition  to  enhancing  the  stu- 
dent's acceptance  of  training  system  deci- 
sions, the  replay  will  provide  the  instruc- 
tor with  an  excellent  tool.  After  the  run, 
the  student  and  instructor  can  review  the 
run  together.  This  will  provide  an  ex- 
cellent forum  for  discussion  of  such  things 
as  subtle  points  of  style. 


Use  of  Synthesized  Speech.  Synthesized 
speech  is  the  ideal  channel  for  some  sys- 
tem outputs  such  as  pilot  responses  and 
instructor  comments.  There  are  other  situa- 
tions in  which  it  may  be  necessary  to  use 
it  with  discretion.  For  example,  many  per- 
sons adapt  to  its  slightly  artificial  sound 
easily,  but  some  express  annoyance  at  hav- 
ing to  listen  to  it  for  long  periods  of 
time  and,  therefore,  its  usefulness  for  pre- 
sentation of  instructional  materials  is 
limited  by  user  acceptance  considerations. 
The  GCA-CTS  design  supports  multimedia 
presentations  of  such  materials,  and  syn- 
thesized speech,  employed  sparingly,  is  ex- 
pected to  enhance  these  presentations. 

Use  of  the  speech  synthesizer  could  in- 
fluence user  acceptance  in  an  indirect  way 
as  well.  Verbal  prompting  is  employed  as 
an  effective  teaching  technique  in  present 
controller  training.  The  laboratory  system 
also  provides  prompts  using  the  speech  syn- 
thesizer, but  there  is  some  debate  as  to 
whether  or  not  synthesizer  prompting  is 
effective  for  good  reference  pattern  col- 
lection. Some  observe  an  unwilling  mimicry 
of  the  speech  synthesizer's  inflection  and 
timing.  Others  do  not  notice  this  effect. 
One  thing  is  certain,  reference  patter,  $ 
created  from  unnatural  verbalizations  do  not 
provide  the  best  possible  recognition  ac- 
curacy. The  efficacy  of  the  use  of  synthe- 
sized prompts  in  the  collection  of  reference 
patterns  will  be  carefully  considered  during 
GCA-CTS  development. 
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SUMMARY 

There  are  many  challenging  problems 
facing  the  system  designer  in  the  application 
of  the  automated  speech  technologies  to  simu- 
lation. We  have  attempted  to  show  that  ef- 
fective strategies  can  be  designed  to  employ 
these  technologies  in  a training  environment. 


These  strategies  include  techniques  for  aug- 
menting shortcomings  both  in  the  state  of  ttie 
art  and  in  user  performance  to  produce  a sys- 
tem which  can  respond  intelligently,  be  ac- 
ceptable to  the  user,  and  can  provide  excel- 
lent training.  These  concepts  will  soon  be 
ready  for  testing,  and  we  look  forward  to  a 
successful  field  evaluation. 
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ABSTRACT 


This  paper  discusses  a test  approach  for 
determining  the  optimum  field  of  view  of  a flight 
simulator  visual  system  as  a function  of  aircraft 
mission.  The  program  is  intended  to  provide  the 
information  needed  to  decide  which  portions  of  the 
aircraft's  total  field  of  view  should  be  simulated 
and  which  can  be  disregarded.  The  approach  consists 
of  three  phases:  drawing  board  evaluation,  aircraft 
ground  testing,  and  flight  testing.  The  procedure  is 
described  using  the  CH-46  helicopter  as  an 
illustrative  example  but  it  is  applicable  directly  to 
any  aircraft,  fixed  or  rotary  wing.  Pilot  qualitative 
ratings  are  obtained  for  each  simulated  visual 
configuration  during  specified  flight  maneuvers. 
Results  from  the  CH-46  program  indicated  that  a 
pilot/copilot  oriented  visual  system  limited  the 
training  potential  for  both  crewmembers.  The 
optimum  configuration  for  the  CH-46E  operational 
flight  trainer  was  a pilot  oriented  visual  system. 

BACKGROUND  AND  PURPOSE 


1.  Following  the  lead  of  commercial  aviation,  the 
U.S.  Navy  is  rapidly  expanding  its  use  of  flight 


trainers  (Operational  Flight  Trainers  and  Weapons 
System  Trainers)  for  pilot  training  and  proficiency. 
This  has  been  made  possible  by  improved  flight 
fidelity  achieved  through  accurate  duplication  of 
actual  aircraft  characteristics  and  the  addition  of 
peripheral  equipment  such  as  audio  systems,  motion 
bases,  and  more  recently,  visual  systems. 

2.  A visual  system  can  significantly  expand  the 
types  and  quality  of  training  possible  in  a flight 
simulator.  For  example,  a realistic  visual  system 
provides  the  potential  for  training  in  the  takeoff  and 
landing  regimes  of  flight  — two  areas  of  high  pilot 
workload  and  increased  accident  potential.  How- 
ever, this  increased  capability  is  not  realized  with- 
out cost  as  a visual  system  represents  a significant 
percentage  of  the  total  expense  of  a trainer. 


3.  There  are  several  types  of  visual  systems  for 
flight  simulators.  Each  has  its  own  particular 
advantages  and  disadvantages,  as  summarized  in 
table  1. 


TABLE  1. 


Type 

Description 

Major 

Advantages 

Major 

Disadvantages 

Primary  Areas 
of  Applications 

Computer  Generated  Image 

Night/Dusk  System 
( Calligraphic) 

TV  type  picture  gen- 
erated by  a computer 
and  special  purpose 
digital  processing 
equipment  from  a 
mathematical  model. 

High  fidelity  with 
real-world 

Low  initial  cost 
(1/2  to  1 Million) 

Low  operating  cost 

Wide  FOV  with 
multiple  displays 

Scene  changes  made 
with  software  changes 

Low  light  level  display 

Day /Night  System 
(Raster  Scan) 

Wide  FOV  with  multiple 
displays 

Scene  changes  made  with 
with  software  changes 
More  scene  content 
capability  than  night/ 
dusk  system 

High  initial  cost 
(3  to  4 Million) 

Reduced  fidelity  in 
day  scene  compared 
with  real-world 

Moderate  operating 
cost 

Takeoff  and  landing 

Camera  Model 

Closed  circuit  TV 
picture  of  a scaled 
model  of  the  problem 
area 

High  scene  detail 

Limited  FOV  and 
problem  area 

Limited  depth  of 
focus 

Moderate  initial 
cost  for  single 
channel  (1  ta  3 

Million) 

Moderate  to  high 
operating  cost 

Low  altitude  slow 
speed  maneuvers 
(Nap-of-the-earth 
flight) 

Area  of  Interest 

Camera  model  with 
high  resolution  target 
imagery  superimposed 
on  a low  resolution 
sky/earth  background 

Almost  unlimited  FOV 

High  resolution  for 
target  imagery 

High  initial  cost 
(3  to  4 Million) 

High  operating  cost 

Air  to  air  combat 
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4.  Ideally,  the  pilot  flying  a simulator  would  be 
presented  a field  of  view  (FOV)  identical  to  that  of 
the  actual  aircraft.  However,  due  to  cost  and 
hardware  limitations,  current  systems  only  present  a 
limited  portion  of  this  total  FOV.  Program  managers 
must  assess  the  tradeoffs  between  training  effec- 
tiveness and  cost  of  a visual  system  when  establish- 
ing the  FOV  requirements.  Test  and  evaluation 
personnel  can  provide  helpful  information  by  deter- 
mining which  portions  of  the  total  FOV  must  be 
simulated  and  which  could  be  eliminated,  based  upon 
mission  requirements.  This  paper  describes  a test 
and  evaluation  method  to  give  the  program  manager 
the  information  he  needs  to  make  an  informed 
decision.  No  attempt  is  made  to  discuss  in  detail 
the  advantages/disadvantages  of  visual  system  type 
or  the  effect  of  scene  content  and  quality  on  pilot 
performance.  Visual- system  fidelity  is  discussed  in 
detail  in  references  (a)  and  (b). 

COMPUTER-GENERATED  IMAGERY  VISUAL 
SYSTEM 

5.  The  test  approach  presented  in  this  paper 
applies  directly  to  any  visual  system  displaying  less 
than  the  actual  aircraft  FOV.  However,  a 
Computer-Generated  Imagery  (CGI)  system  is  used 
as  the  basis  for  discussion  since  the  CH-46E 
Operational  Flight  Trainer  (OFT)  used  a CGI  sys- 
tem. A brief  explanation  of  the  CGI  system  is 
presented  here  to  introduce  terminology  which  is 
used  later  in  this  paper. 


8.  The  number  of  channels  that  can  be  used  for 
a visual  system  is  limited  by  the  system's  computer 
capacity  and  the  size  of  the  displays  and  supporting 
structures.  Increasing  the  number  of  channels  de- 
creases the  average  scene  content  of  each  channel, 
as  a given  computer  is  capable  of  generating  only  a 
finite  number  of  edges,  and  these  must  be  divided 
among  the  channels.  The  fewer  edges  there  are  in  a 
scene  the  less  detailed  and  more  cartoonish  it  will 
appear.  Repeater  windows  which  give  the  same 
picture  as  another  require  no  additional  computation 
capability,  and  in  some  applications,  can  be  used  to 
give  the  copilot  or  instructor  a visual  display 
without  sacrificing  scene  quality. 

DETERMINING  FIELD  OF  VIEW  REQUIREMENTS 

GENERAL 

9.  If  program  cost  restraints  dictate  a number  of 
data  channels  and  displays  less  than  required  to 
simulate  the  full  aircraft  FOV,  then  the  optimum 
arrangement  of  visual  displays  should  be  determined 
by  the  flight  test  program  presented  in  this 
technical  paper.  With  results  from  such  an 
approach,  the  program  manager  will  be  able  to  make 
meaningful  decisions  on  cost  and  required  visual 
system  capabilities.  In  addition,  squadron  personnel 
will  be  able  to  determine  what  mission  tasks  should 
be  included  in  their  training  syllabus.  The  amount  of 
training  degradation  with  limited  displays  can  also 
be  determined. 


6.  For  a CGI  system,  a mathematical  model  of  a 
particular  visual  scene  is  programmed  into  a com- 
puter. This  scene  must  be  mathematically  defined  in 
terms  of  a finite  number  of  straight  line  segments  or 
edges.  The  larger  the  capacity  of  the  computer,  the 
greater  the  number  of  edges  available  and,  there- 
fore, the  more  detailed  the  scene  can  be.  The 
computer  output  is  directed  through  Cathode-Ray 
Tubes  (CRT's),  similar  to  television  picture  screens. 
These  CRT's  are  called  windows,  which  is  what  they 
appear  to  be  to  the  pilot  in  the  simulator.  The 
number  of  separate  scenes  the  computer  can  gener- 
ate simultaneously  determines  the  number  of  chan- 
nels a system  can  have.  Thus,  a simple  system  with 
two  CRT's,  one  for  the  pilot  and  one  for  the  copilot, 
each  showing  the  same  identical  picture,  would  be  a 
two-window,  one-channel  system.  The  copilot's 
window  requires  no  additional  computer  capacity  as 
its  scene  is  just  a repeat  of  the  information  put  into 
the  pilot's  window.  This  two-window,  one-channel 
system  is  called  a pilot/copilot  visual  system,  since 
the  focal  point  of  one  display  is  the  pilot's  eye 
position  and  the  focal  point  of  the  other  display  is 
the  copilot's  eye  position.  The  pilot  is  not  able  to  see 
the  copilot's  display  nor  is  the  copilot  able  to  see  the 
pilot's  display. 

7.  A visual  system  with  two  CRT  displays,  each 
showing  different  scenes,  is  a two-window, 
two-channel  system.  If  the  focal  point  of  both 
displays  is  the  pilot's  eye  position,  the  system  is 
called  a pilot-oriented  or  pilot-only  visual  system.  A 
display  located  in  the  copilot's  front  windscreen  in  a 
pilot-oriented  system  provides  a distorted  scene  to 
the  copilot. 


10.  The  mission  and  operating  environments  of  the 
aircraft  have  an  impact  on  the  visual  system  FOV 
requirements.  The  FOV  requirements  can  vary 
significantly  among  simulators  for  different  aircraft 
due  to  the  different  mission  environments  of  the 
aircraft.  For  an  Anti-Submarine  Warfare  helicopter, 
adequate  FOV  representation  of  the  shipboard 
landing  task  is  essential  for  proper  training. 
Reference  (b)  discusses  an  evaluation  of  the  SH-ZF 
Weapons  System  Trainer  (WST)  where  there  were 
training  limitations  due  to  inadequate  FOV.  For  a 
Marine  assault  helicopter,  duplication  of  the 
shipboard  environment  is  also  important,  but  the 
primary  emphasis  would  be  placed  on  FOV  consider- 
ations relative  to  typical  landing  zones,  particularly 
those  in  a confined  area.  Reference  (c)  discusses  the 
evaluation  conducted  to  determine  the  FOV  simula- 
tion requirements  for  the  CH-46E  OFT. 

TEST  APPROACH 

11.  The  test  approach  for  evaluating  simulator  FOV 
requirements  is  logically  divided  into  three  phases: 
drawing  board,  ground  test,  and  flight  test.  In  these 
phases,  configurations  of  candidate  window  displays 
are  evaluated.  As  the  program  progresses  through 
these  stages,  changes  to  display  configurations 
become  increasingly  difficult,  so  attention  given  to 
the  early  stages  can  result  in  payoffs  later. 

Drawing  Board 

1Z.  This  is  essentially  a cut  and  paste  "what  do  you 
think”  sort  of  exercise  in  which  pilots  experienced  in 


1 


34 


the  aircraft  to  be  simulated  and  engineers  familiar 
with  visual  system  characteristics  discuss  possible 
visual  display  groupings  and  decide  upon  a number  of 
candidate  configurations  to  evaluate.  To  accomplish 
this,  diagrams  of  the  aircraft  FOV  from  design 
eyepoint  are  drawn  on  a graph,  which  has  vertical 
and  horizontal  FOV  angles  as  ordinate  and  abscissa, 
respectively.  An  example  of  such  a graph  from  the 
pilot's  design  eyepoint  for  the  CH-46  helicopter  is 
presented  in  appendix  B.  Graphs  of  this  format  can 
be  produced  readily  by  the  FOV  Evaluation  Appara- 
tus (FOVEA)  described  in  appendix  C.  To  evaluate  a 
visual  system  for  the  copilot,  FOV  graphs  must  be 
created  with  the  FOVEA  from  the  copilot's  seat 
since  the  presentation  may  not  be  the  same  from  the 
different  design  eyepoint.  The  pilots  arrange  cutouts 
of  the  candidate  window  displays  on  the  FOV  graph. 
The  engineers  advise  the  group  as  to  the  technical 
feasibility  of  the  proposed  configurations,  as  limited 
by  physical  dimensions  of  the  units,  necessary 
overlap  for  adjacent  units,  or  potential  alignment 
problems.  Generally,  each  pilot  will  have  a different 
opinion  on  the  relative  importance  of  certain 
windows.  This  is  due  partly  to  individual  differences 
in  pilot  technique  and  partly  to  the  variety  of 
missions  each  pilot  has  flown.  Thus,  if  a multi- 
mission aircraft  is  being  simulated,  pilots  experi- 
enced in  each  of  the  missions  should  be  included  in 
the  evaluation.  However,  the  evaluation  group 
should  be  kept  small  enough  to  allow  each  pilot's 
opinion  to  be  weighed  and  still  come  to  an  agree- 
ment within  a reasonable  amount  of  time.  Once  the 
candidate  display  configurations  have  been  decided, 
it  is  appropriate  to  begin  working  directly  with  the 
aircraft. 


Ground  Testing 

13.  The  first  step  in  the  ground  test  phase  is  to 
draw  the  candidate  window  display  configurations  on 
the  aircraft  windscreen.  The  FOVEA  is  very  useful 
in  sighting  and  documenting  the  appropriate  window 
locations.  Once  a complete  window  display  arrange- 
ment is  sketched,  the  pilots  make  a preliminary 
appraisal.  Generally,  a number  of  minor  relocations 
or  adjustments  are  desired  at  this  stage  and  they  can 
still  be  accomplished  with  relative  ease.  At  this 
stage  it  is  also  possible  (and  highly  desirable)  to 
eliminate  some  of  the  candidate  configurations. 
From  the  sketches  on  the  windscreen,  amber  cello- 
phane (AMBERLITH)  is  cut  into  shapes  to  fit  the 
canopy  with  openings  where  the  proposed  windows 
would  be.  An  example  used  in  the  CH-46E  OFT  pro- 
gram is  presented  in  appendix  D.  Using  blue  lens 
goggles  (Device  l-F-4-b),  the  pilots  can  get  a good 
idea  of  what  the  corresponding  simulator  FOV  will 
be.  The  amber  portions  of  the  canopy  windscreen 
will  appear  black  while  the  pilot  will  be  able  to  see 
through  the  display  window  cutouts  and  determine 
the  suitability  of  each  configuration.  Modification  to 
the  configurations  now  becomes  increasingly 
difficult  but  will  continue  to  be  necessary  if  an 
optimum  arrangement  is  to  be  obtained.  At  this 
stage,  it  is  still  feasible  to  have  a relatively  large 
number  of  pilots  participate  in  the  evaluation,  but  in 
the  flight  phase,  time  and  financial  constraints  will 
probably  limit  the  number  of  evaluators. 


Flight  Testing 

14.  The  final  phase  in  determining  the  FOV  require- 
ments is  to  fly  the  aircraft  with  a minimized  number 
of  candidate  window  display  configurations.  Each 
configuration  should  be  evaluated  by  at  least  two 
pilots  for  a series  of  specified  mission  tasks.  An 
example  of  appropriate  mission  tasks  for  the 
CH-46E  medium  assault  helicopter  is  contained  in 
the  evaluation  sheet  presented  in  appendix  E.  The 
acceptability  of  the  FOV  in  accomplishing  these 
tasks  is  rated  by  the  pilot  on  a scale  of  0 to  5,  when 
0 represents  the  degree  of  difficulty  in  performing  a 
maneuver  with  no  windows  at  all  or  as  would  be 
typical  in  instrument  meteorological  conditions  and 
5 represents  the  degree  of  difficulty  in  performing  a 
maneuver  with  the  actual  aircraft  FOV.  Typically, 
each  configuration  will  show  strengths  for  some 
tasks  and  weaknesses  for  others.  After  flying  each 
configuration,  the  results  are  evaluated  and  one  of 
the  configurations,  or  more  likely  a modification  of 
it,  is  selected  as  the  best.  Time  should  be  allocated 
in  the  test  program  to  fly  this  "final"  configuration 
to  determine  if  it,  in  fact,  offers  everything 
expected  of  it.  The  project  evaluation  team  will 
then  be  confident  that  the  recommended  configura- 
tion offers  the  best  possible  training  potential  within 
the  constraints  allowed. 


FOV  Considerations 

15.  As  discussed  in  paragraphs  6 and  7,  the  visual 
system  may  be  pilot  or  pilot/copilot  oriented.  If  the 
decision  for  a one-pilot  or  two-pilot  system  is  not 
made  during  the  "drawing  board"  or  ground  test 
phases,  it  should  be  accomplished  early  in  the  flight 
program.  It  is  not  satisfactory  to  orient  front 
windscreen  displays  for  both  pilot  and  copilot.  A 
single  front  display  will  limit  the  copilot  to  forward 
flight  profiles  with  little  capability  to  fly  curved 
approaches  or  to  perform  precision  flight  requiring 
outside  periphial  visual  reference.  Reference  (c) 
reported  that  a pilot/copilot  system  for  the  CH-46E 
OFT  limited  both  the  pilot  and  copilot.  The  configu- 
ration consisting  of  a copilot  front,  copilot  left  side, 
and  pilot  right  side  was  described  by  the  copilot  as 
more  frustrating  than  useful  when  performing 
typical  H-46  missions,  with  the  single  exception  of  a 
straight-in  instrument  approach  to  a roll-on  landing. 

16.  An  adequate  vertical  FOV  is  essential  for  a 
helicopter  trainer.  The  lower  edge  of  the  front 
displays  should  be  close  to  the  bottom  of  the 
windscreen  for  adequate  look-down  capability  during 
steep  approaches  and  flares,  precision  hover,  slow 
speed  In  Ground  Effect  flight,  and  ship  landings. 
The  initial  "drawing  board"  position  of  the  pilot's 
front  display  for  the  CH-46E  OFT,  reference  (c), 
was  shifted  downward  to  enable  the  pilot  to  keep 
sight  of  the  landing  spot.  The  restricted  FOV  of  the 
SH-2F  WST,  particularly  in  the  lower  look-down 
angles,  created  problems  in  precise  hover,  low-  speed 
flight,  and  frigate  landings  as  reported  in  reference 
(b).  Reference  (c)  reported  that  flight  test  results 
verified  the  pilot  chin  window  display  was  important 
in  the  proposed  H-46  OFT  visual  system  when  the 
helicopter  was  near  the  ground  and  moving. 
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17.  Items  that  should  be  considered  for  the  top  of 
the  displays  for  helicopter  OFT's  include  the  horizon 
in  forward  flight  and  the  rotor  tip  path  plane  for 
ground  operation  or  confined  area  landings. 
Simulation  of  maneuvers  involving  large  forward 
control  inputs  and  typical  autorotative  entries  re- 
quire a FOV  display  above  the  normal  level  flight 
horizon. 

18.  Horizontal  FOV  requirements  are  also  dictated 
by  mission  requirements.  Flight  operations  requiring 
360  deg  turning  patterns  (i.e.,  plane  guard,  Magnet- 
ic Anomaly  Detection  trapping,  etc.)  or  both  left- 
and  right-hand  shipboard  approaches  with  traffic 
monitor  will  require  substantially  more  horizontal 
FOV  than  forward  flight  cruise  and  straight-in 
landings.  Visual  information  aft  of  the  pilot  or 
copilot  90  deg  position  in  the  CH-46  provided  only 
low  priority  information,  as  reported  in  refer- 
ence (c). 

Safety  Precautions 

19.  As  in  any  flight  test  program,  certain  safety 
considerations  should  be  addressed  during  the  flight 
test  phase.  The  amber  cellophane  installed  in  the 
windscreen  of  the  aircraft  causes  negligible 
degradation  of  visibility  in  clear  or  cloudy  daytime 
light  conditions  but  seriously  hampers  the  pilot's 


visibility  at  night.  Therefore,  these  flights  should 
not  be  flown  at  night.  Also,  the  safety  pilot  should 
not  use  sunglasses  or  a shaded  visor  during  the 
evaluation  flights  since  these  also  restrict  visibility 
through  the  amber  cellophane.  Since  some  window 
display  candidate  configurations  may  severely  re- 
strict FOV  in  some  quadrants,  the  crew  chief  should 
be  specifically  briefed  to  pay  particular  attention  to 
these  quadrants  and  advise  the  pilots  of  traffic  or 
obstacles. 

SUMMARY 

20.  This  test  approach  for  determining  flight  train- 
er visual  system  FOV  requirements,  described  using 
the  CH-46E  OFT  as  an  illustrative  example,  can  be 
applied  directly  to  any  aircraft,  either  fixed  or 
rotary  wing.  It  provides  the  evaluation  team,  pro- 
gram manager,  and  squadron  personnel  with  infor- 
mation on  the  impact  of  FOV  on  mission  training 
potential.  There  are  other  visual  system  considera- 
tions beyond  the  scope  of  this  type  of  evaluation 
which  have  a significant  impact  on  visual  system 
fidelity  and  training  potential.  Among  these  are 
scene  content  and  quality,  image  realism,  aliasing 
tendencies,  and  visual  system/trainer  interface. 
Defining  simulator  FOV  requirements  is  only  one 
part  of  the  problem  but  an  important  early  step  in 
the  process. 
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APPENDIX  C 

FIELD  OF  VIEW  EVALUATION  APPARATUS 

1.  The  Field  of  View  Evaluation  Apparatus 
(FOVEA)  is  an  instrument  capable  of  measuring, 
recording,  and  graphically  plotting  the  angular 
subtense  of  objects  relative  to  the  design  eye 
reference  point  of  crew  stations.  Its  principal 
application  is  the  measurement,  for  evaluation  and 
verification  purposes,  of  the  external  FOV  of 
aircrew  stations.  It  is  designed  for  use  in  both 
mock-ups  and  actual  aircraft  crew  stations  and 
generates  the  FOV  plots  required  for  verification  of 
data  submitted  in  accordance  with  MIL-STD-850B. 
It  also  has  utility  in  the  evaluation  of  visual 
obstructions  to  the  line-of-sight  to  controls  and 
displays,  documentation  of  simulator  visual  system 
FOV,  verification  of  control/display  location 
relative  to  the  primary  visual  areas  for  caution/ 
warning  displays,  photographically  recording  from 
the  desired  viewing  position  (design  eye),  and  identi- 
fication of  deficiencies  or  enhancing  characteristics 
during  FOV  evaluation  tests. 

2.  FOVEA  consists  of  a commercially  available 
programmable  desk-top  calculator  and  X-Y  plotter, 
a servo-driven  TV  sensor  element,  a TV  monitor,  a 
joystick  for  sensor  pointing  control,  and  associated 
electronics  on  adapter  brackets  for  positioning  the 
sensor  in  the  crew  station  seat.  A magnetic  tape 
cassette  recorder  and  hard-copy  printer  are  integral 
features  of  the  calculator  and  provide  for  data 
storage  and  hard  copy  printout  of  measured  data. 
The  system  software  provides  for  initial  setup 
alignment,  sensor  positioning  error  compensation, 
and  selection  of  plotter  output  options  of  rectilinear 
plots,  equal  area  projection,  and  tangent  plots  for 
generation  of  landing  vision  data.  A polaroid  camera 
is  included  to  obtain  photographic  records  of  the  TV 
monitor  display. 

3.  Test  setup  requires  placement  of  the  TV  sensor 
on  the  crew  station  seat  pan  (figure  1)  and  adjust- 
ment of  the  sensor  to  coincide  with  the  design  eye 
location  for  the  crew  station.  Alignment  procedures 
verify  the  accuracy  of  sensor  positioning.  The  sensor 
pointing  angle  is  controlled  remotely  by  the  joystick 
control  (figure  2).  Data  were  acquired  by  the  opera- 
tor tracing  the  outlines  of  visual  obstructions  (e.g., 
canopy  rails,  windscreen  bracketry,  instrument 
panel)  by  tracking  with  the  center  of  the  FOVEA 
sensor,  represented  by  a cursor  on  the  TV  monitor. 
After  data  acquisition,  the  desired  output  options 
are  selected  via  controls  on  the  calculator  and  the 
data  are  presented  graphically  on  the  X-Y  plotter. 
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Figure  1 

FOVEA  TV  Camera  Mounted  in 
Pilot's  Seat  of  H-46  Helicopter 
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Figure  2 

FOVEA  Control  Equipment  Positioned  in 
H-46  Helicopter  Cabin 
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APPENDIX  E 

SAMPLE  MISSION  TASK  EVALUATION  SHEET 
(USED  IN  CH-46E  OFT  VISUAL  SYSTEM  EVALUATION) 


Aircraft  Type 

BuNo 

Time  T.O. 
Time  Land 

Crew  Name 

Weight 

Station 

Visual  Test  C 

Configuration 

Weather 

Wind 

Visibility 

Maneuver 

Ground  Taxi 

Forward 

Aft 

Left 

Right 

Hover 

Air  Taxi 

Forward 

Aft 

Left 

Right 

Forward  Flight 

Level 

Left  Bank 
Right  Bank 

Confined  Area  Landing 

Normal  Approach 


Precision  Approach 


LPH  Approach 


360  deg 

Left  Turn 
Right  Turn 

Autorotative  Approach 

Left 

Right 

Autorotative 

Entry 

Recovery 

Quick  Stop 

External  Pickup 


External  Drop 
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COMPUTER  IMAGE  GENERATION  OF  CURVED 
OBJECTS  FOR  SIMULATOR  DISPLAYS 


DR.  GEOFFREY  Y.  GARDNER 
Research  Scientist 
Grumman  Aerospace  Corporation 


INTRODUCTION 

Vehicle  simulators  have  become  a stan- 
dard tool  in  the  training  of  aircraft  pilots 
and  other  vehicle  operators  because  use  of 
the  actual  vehicle  can  be  prohibitively 
expensive  and  dangerous.  In  addition,  train- 
ing scenarios  that  might  be  undesirable  to 
represent  in  reality  can  be  simulated  in  a 
controlled  environment.  Thus,  with  simula- 
tors, pilots  can  train  for  air-to-air  combat 
or  missile  evasion  and  can  practice  repeated 
"landings"  on  an  aircraft  carrier  in  heavy 
seas.  In  this  way,  astronauts  were  able  to 
become  proficient  at  landing  on  the  moon 
long  before  the  lunar  lander  had  reached  the 
launch  pad. 

Among  the  most  important  training  cues 
in  the  vehicle  simulator  is  the  visual  dis- 
play. Traditionally,  simulation  of  motion 
has  been  provided  by  large  model  board 
systems  viewed  through  television.  As  the 
trainer  pilot  controls  his  vehicle  motion  in 
"simulation  space,"  a TV  camera  ranges  over 
the  scene  on  the  model  board  and  transmits 
a view  corresponding  to  his  simulated  per- 
spective. In  general,  however,  the  model 
board  technique  has  been  found  to  be  too 
limiting.  To  cover  the  large  ground  areas 
needed  for  high-speed  aircraft  simulation,  a 
model  board  of  unwieldy  size  would  be  re- 
quired. A change  of  scene  requires  an 
entirely  new  model  board,  and  board  construc- 
tion and  alteration  are  very  expensive. 

There  are  also  optical  problems  such  as 
depth  of  focus.  Finally,  certain  physical 
scenarios  are  awkward  or  impossible  with 
model  boards.  Low-level  hel icopter  fl ight 
among  trees,  actual  touchdown  and  motion 
under  objects  are  examples.  Bombing  missions, 
missile  firing,  and  refueling  are  others. 

The  basis  of  all  these  problems  lies  in  the 
physical  nature  of  the  model  board  scene. 

Computer  Image  Generation  (CIG)  avoids 
these  problems  by  using  an  abstract  scene 
stored  in  a computer  memory  in  mathematical 
form.  Many  scenes  can  be  stored,  and  changes 
can  be  made  easily.  Any  environment  can  be 
simulated,  and  optical  problems  are  non- 
existent. A pilot  can  fly  under  bridges, 
fire  weapons,  bomb  portions  of  the  scene, 
and  even  crash  without  consequence.  For 
these  reasons,  CIG  is  becoming  more  and  more 
the  standard  method  for  simulator  displays. 


STATE  OF  THE  ART  IN  CIG 

The  basic  problem  with  CIG  is  that  dy- 
namic displays  require  high-frame  rates, 
necessitating  the  production  of  an  entire 
picture  every  thirtieth  of  a second.  For 
complex  scenes,  this  means  that  massive 
amounts  of  computation  must  be  accomplished 
in  a very  short  time.  From  a large  global 
scene  stored  in  computer  memory,  the  portion 
visible  to  a pilot  in  a given  frame  time 
must  be  determined,  transformed  according 
to  the  pilot's  perspective,  and  projected 
onto  the  image  plane  (as  realized  by  the  TV 
monitor).  All  this  must  be  done  continuously 
at  TV  frame  rates  and  in  accordance  with  the 
raster  scan  format  of  the  monitor.  All  hid- 
den surface  portions  must  be  properly 
occluded  and  point-by-point  illumination  of 
each  visible  surface  realistically  repre- 
sented. Any  time  delay,  computation  error, 
or  misrepresentation  of  a scene  feature 
will  adversely  affect  the  simulation  and  the 
training. 

To  accomplish  the  high-computation  rates 
needed,  the  current  state  of  the  art  in  CIG 
employs  a technique  we  will  call  the  "Planar 
Patch  Method."  This  technique  simplifies 
the  problem  by  reducing  any  arbitrary  scene 
to  an  approximation  consisting  of  straight 
edges  and  flat  surface  patches  (Ref.  1). 

The  advantage  to  the  Planar  Patch  Method  is 
that  it  reduces  computations  to  the  simplest 
level  possible.  Since  location  of  all 
projected  edge  points  are  calculated  by 
intersection  with  the  scan  lines,  the  tech- 
nique need  handle  only  simple  intersections 
between  straight  lines.  Since  all  surfaces 
are  flat,  shading  along  a surface  between 
edge  points  can  be  filled  in  with  a constant 
value.  If  the  scene  being  modeled  consists 
totally  of  flat  objects,  such  as  a runway 
scene  with  DaralleleoiDed  buildings,  the 
Planar  Patch  Method  provides  realistic  simu- 
lation. Furthermore,  if  a limited  number  of 
edges  are  sufficient  to  describe  the  scene, 
this  technique  can  achieve  TV  frame  rates  by 
use  of  special-purpose  hardware  arrayed  to 
"pipeline"  the  many  simple  computations. 

The  problem  with  the  Planar  Patch 
Method  is  that  it  is  not  designed  to  model 
curved  surfaces.  Any  representation  of  a 
curved  object  by  a finite  number  of  straight 
edges  and  planar  patches  can  only  be  an 
approximation  (Fig.  1).  The  appearance  of 
artificial  edges  and  shading  discontinuities 
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computed.  The  Bicubic  Patch  Method,  there- 
fore, has  not  proven  applicable  to  real-time 
CIG. 


Figure  1.  Planar  Patch  Representation 
of  a Cylinder 

in  supposedly  smooth  surfaces  (such  as  oil 
tanks  or  aircraft  fuselages)  degrades  the 
visual  simulation  to  an  unacceptable  level. 

To  avoid  this,  two  adjustments  must  be  made. 
First,  a disproportionate  number  of  edges 
must  be  assigned  to  each  curved  object,  and 
second,  shading  discontinu . ties  between 
planar  patches  must  be  smoothed  over  by 
interpolation.  The  result  is  increased 
complexity  in  both  the  data  base  and  the  com- 
putation algorithms.  Thus  the  Planar  Patch 
Method,  which  represents  a simplification  in 
the  generation  of  flat  images,  must  back- 
track into  increasing  complexity  to  generate 
curved  surfaces.  It  should  also  be  noted 
that  no  matter  how  well  the  shading  is 
smoothed  to  hide  edges  interior  to  the 
object  silhouette  boundary,  the  boundary  it- 
self will  always  show  an  "edginess"  for  any 
finite  number  of  edges  used. 

To  overcome  the  problems  inherent  in  the 
Planar  Patch  Method  and  to  advance  the  state 
of  the  art  in  CIG,  the  Research  Department 
of  the  Grumman  Aerospace  Corporation  has 
undertaken  a project  to  develop  techniques 
that  will  allow  realistic,  real-time  gener- 
ation of  images  of  curved  objects. 

CIG  OF  CURVED  OBJECTS 

There  are  two  techniques  currently  in 
use  for  generating  realistic  images  of  curved 
surfaces.  The  first  technique  is  the  Bicubic 
Patch  Method,  which  models  a curved  object 
by  a set  of  third-order  surface  patches. 

Work  in  this  area  has  been  pursued  at  the 
University  of  Utah  (Ref.  2)  and  produces 
very  realistic  renderings  of  sculptured  sur- 
faces such  as  bottles  and  wine  glasses. 
Because  of  the  complexity  of  the  bicubic 
patch  equations,  however,  the  data  base  is 
extensive  and  complicated.  More  importantly, 
each  image  picture  element  must  be  examined 
and  a costly  subdivision  of  each  patch 
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The  second  curved  surface  technique, 
which  we  call  the  Ray  Tracing  Method  (RTM), 
uses  second-order  surfaces  to  define  objects 
(Ref.  3).  Whole  objects  can  be  modeled  by 
only  a few  such  surfaces,  sometimes  by 
just  one  second-order  surface  with  one  or  two 
planar  surfaces  (Fig.  2).  Because  the 
second-order  surface  is  the  simplest  surface 
that  can  model  true  curvature,  and  because 
so  few  surfaces  are  needed  to  represent 
objects,  the  RTM  offers  the  potential  for 
the  simplest  data  base  and  simplest  compu- 
tation load  for  generating  curved  surface 
images.  For  this  reason,  we  have  used  the 
RTM  as  the  basis  for  our  study. 


WING 


Figure  2.  Ray  Tracing  Method  (RTM) 
Representation  of  a Wing 

The  basic  RTM  algorithm  works  as  follows. 
A scene  space  consisting  of  an  arbitrary 
number  of  objects  each  composed  of  first-  and 
second-order  surfaces  is  defined  in  computer 
memory  along  with  a defined  illumination 
vector  (Fig.  3).  From  a specified  eye  point, 
a ray  is  traced  through  each  pixel  in  the 
image  plane.  A test  is  made  for  intersection 
of  the  ray  with  each  surface  of  each  object 
in  the  scene  space.  The  point  on  the  sur- 
face nearest  to  the  eye  point  is  the  point 
visible  at  the  given  pixel.  Shading  inten- 
sity is  determined  by  calculating  the  surface 
normal  at  the  visible  point  and  computing  the 
cosine  between  the  normal  and  the  illumina- 
tion vector.  Since  a typical  scene  would 
require  many  objects  and  would  be  presented 
on  a display  consisting  of  over  500  points 
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on  over  500  scan  lines,  it  can  easily  be  seen 
that  the  brute  force  RTM  requires  far  too 
much  computation  to  achieve  real-time  rates. 
However,  inherent  in  the  basic  concept  lies 
the  potential  for  drastically  reducing  the 
number  of  calculations. 


Figure  3.  RTM  Schematic  Diagram 
MODIFIED  RTM  FOR  REAL-TIME  CIG 

We  have  made  major  modifications  to  the 
basic  RTM  which  eliminate  the  pixel-by-pixel 
ray  tests  and  thereby  reduce  computation  time 
drastically.  Although  the  details  of  our 
algorithms  are  proprietary,  the  essence  is  as 
follows.  We  calculate  only  the  high-order 
scene  information  with  complete  precision 
and  fill  in  the  low-order  information  with 
a smooth  approximation.  Since  the  high-order 
information  represents  only  a small  fraction 
of  the  total  imaqe,  the  effectiveness  of  this 
approach  is  obvious.  The  test  scene  in  Figs. 
4 and  5 illustrates  our  procedure.  The  KC- 
135  is  composed  of  21  objects.  The  fuselage 
includes  5 objects,  portions  of  a cylinder, 
a cone  and  3 ellipsoids;  the  5 aerodynamic 
surfaces  are  each  made  of  a thin  bounded 
elliptical  cone;  each  engine  and  engine  mount 
is  a bounded  ellipsoid;  the  boom  is  a cylin- 
der; and  the  rudderators  are  thin  elliptical 
cylinders. 

Included  in  the  algorithms  are  routines 
to  remove  all  hidden  surface  portions.  The 
visibility  logic  works  on  two  levels.  First, 
all  hidden  surface  portions  on  individual 
objects  are  removed;  then  all  surface  por- 
tions occluded  by  other  objects  are  removed. 


Figure  4.  Modified  RTM  Scene  of  KC-135  Show- 
ing High-Order  Information  Only 


Figure  5.  Modified  RTM  Scene  with  Low-Order 
Information  Filled  in 

The  image  generation  procedure  is  divid- 
ed into  two  parts.  First,  the  frame  pro- 
cessor transforms  the  scene  objects  to  viewer 
perspective,  extracts  the  high-order  infor- 
mation, and  performs  the  intraobject  visibil- 
ity logic.  Then  the  scan-line  processor 
performs  the  interobject  visibility  and  fills 
in  the  low-order  information.  Computation 
time  for  the  boundary  image  in  Fig.  4 was 
two  seconds;  the  shaded  aircraft  image  in 
Fig.  5 took  six  seconds.  The  program  was  run 
on  a general-purpose  minicomputer  (Data 
General  Eclipse)  with  no  special-purpose 
hardware.  A comparison  test  of  the  brute 
force  RTM  took  three  hours  to  generate  the 
shaded  aircraft.  This  represents  three-to- 
four  orders  of  magnitude  improvement  with  our 
refinements. 

Figure  6 shows  a different  view  of  the 
KC-135  scene,  and  Fig.  7 shows  a sequence  of 
images  reDresenting  a refueling  mission.  A 
ground  plane  and  sky  background  have  been 
added  for  more  realism.  It  should  be  noted 
that  the  Planar  Patch  Method  would  require 
over  1000  straight  edges  to  model  the  KC-135 
and  would  result  in  considerable  loss  in 
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realism  as  the  viewer  approached  the  aircraft. 
Indeed,  no  finite  number  of  straight  edges 
could  duplicate  the  fidelity  of  the  KC-135 
refueling  scene. 

NONVISUAL  SENSOR  SIMULATION 


Because  the  RTM  uses  smooth  surfaces,  it 
will  model  nonvisual  sensors  such  as  radar 
with  the  same  data  base  used  for  visual 
images  simply  by  changing  the  illuminance/ 
reflectance  algorithm.  Figure  8 is  a se- 
quence representing  a simulation  of  synthetic 
aperture  radar  images  of  the  KC-135  scene. 
Arbitrary  levels  of  resolution  can  be  modeled 
to  simulate  real  radar  systems.  Note  that 
this  type  of  radar  simulation  would  be  im- 
possible using  the  Planar  Patch  Method  be- 
cause the  large  number  of  surface  discontin- 
uities would  create  an  artificial  scattering 
effect. 


"Scanned"  Image 
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Figure  8.  Synthetic  Aperture  Radar  Simulation 


Because  the  RTM  data  base  consists 
largely  of  whole  volumes  rather  than  surface 
patches,  infrared  (IR)  images  can  be  modeled 
by  a simple  extension  of  the  data  base.  For 
example,  an  oil  tank  could  be  represented 
visually  by  a single  cylinder  section  whose 
image  is  shaded  according  to  the  standard  re- 
flectance model.  For  IR  simulation,  the  data 
base  would  be  extended  to  two  cylinder  sec- 
tions, one  for  the  oil-filled  portion  and  one 
for  the  empty  portion  of  the  tank  (Fig.  9). 
Shading  would  be  determined  according  to  a 
radiance  model,  and  each  of  the  two  tank 
sections  would  have  its  own  radiance  parameter. 
Day-night  changes  would  be  effected  in  the 
same  data  base  by  radiance  parameter  chanoes. 
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OIL  TANK 


IR 
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CYLINDER 
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Figure  9.  Infrared  Simulation 


DISCUSSION 


In  comparison  to  the  Planar  Patch 
Method,  the  modified  RTM  has  the  basic  disad- 
vantage that  the  second-order  surfaces  and 
curves  require  more  complicated  computations. 
This  disadvantage  can  be  minimized  (and  per- 
haps removed  entirely)  by  breaking  down  the 
more  complex  calculations  into  a series  of 
simple  steps  that  can  be  implemented  in 
special-purpose  pipeline  hardware.  We  are 
currently  studying  such  a configuration  for 
the  scan- line  processor  algorithms  as  well  as 
a parallel  array  of  microprocessors  for  the 
frame  processor.  It  must  be  remembered  that 
even  the  Planar  Patch  Method  requires  specially 
designed  hardware  to  achieve  TV  frame  rates. 

In  contrast,  the  advantages  of  the  mod- 
ified RTM  are  numerous.  First,  curvature  is 
represented  faithfully,  no  matter  how  close  the 
viewer  is  to  the  surface.  No  smoothing  is 
required  across  patch  edges  because  there 
are  none.  Secondly,  because  objects  tend  to 
be  represented  as  whole  bodies,  the  data  base 
is  smaller,  simpler,  and  easier  to  construct. 
Thirdly,  because  surfaces  are  represented 
smoothly  without  discontinuities,  nonvisual 
sensors  such  as  radar  can  be  modeled  using 
the  same  data  base. 

FUTURE  WORK 

We  are  developing  a conceptual  design  of 
special-purpose  hardware  and  microprocessors 
to  implement  the  algorithms  at  TV  frame  rates. 
We  will  incorporate  refinements  to  handle 
more  complex  objects  and  algorithms  to  gen- 
erate texture. 
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THE  AWAVS  DATA  BASE  FACILITY  - A COMPREHENSIVE 
PREPARATION  PACKAGE 
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INTRODUCTION 

A complete  data  base  creation  facility  has 
been  developed  by  General  Electric  for  the 
Aviation  Wide-Angle  Visual  System  (AWAVS) 
computer  image  generator  (CIG)  at  the  Naval 
Training  Equipment  Center  in  Orlando.  This 
facility  provides  both  on-  and  off-line  capabilities 
to  attack  all  facets  of  the  data  base  development 
problem.  While  an  off-line  data  base  preparation 
device  alone  will  improve  the  initial  ability  to 
digitize  and  create  visual  data  bases,  the  task  is 
incomplete  and  provision  must  be  made  for  rapid 
on-line  viewing  and  alteration  of  a new  data  base 
to  achieve  successful  environments  in  a reason- 
able length  of  time.  An  off-line  facility  without 
on-line  support  tools  is  like  designing  an  air- 
craft without  a wind  tunnel.  Results  are  known 
only  when  it  is  flown,  and  any  modifications  must 
be  done  statically  (off-line). 

REVIEW 

Too  often,  CIG  systems  have  turned  Into  a 
fly  now,  pay  later  operation  when  an  attempt  is 
made  to  modify  the  existing  operational  data 
bases  or  add  new  training  environments.  The  pay 
aspect  takes  the  shape  of  two  major  costs;  one, 
the  slow  turnaround  time  of  developing  and 
refining  new  data  bases  to  a usable  form;  the 
second  is  system  time  required  on  the  on-line 
computer  for  actually  running  the  off-line  soft- 
ware and  correcting  programming  and  data 
errors.  Consequently,  CIG  system  procure- 
ments are  increasingly  recognizing  the  impor- 
tance of  having  some  type  of  data  base  facility  to 
support  full-time  utilization  of  the  capabilities 
inherent  in  a completely  configured  CIG  system. 

The  AWAVS  system  is  a two-channel  CIG 
system  with  many  developmental  features  needed 
in  the  research  environment  for  which  it  was  in- 
tended. One  of  the  unique  features  of  this  system 
is  the  data  base  development  facility  which  is  the 
subject  of  this  paper.  Recognition  of  require- 
ments for  a complete  set  of  data  base  creation 
tools  resulted  in  the  design  of  the  first  data  base 
creation  package  incorporating  not  only  off-line 
equipment  and  software,  but  also  complementary 
on-line  hardware  additions  which  allow  effective 
utilization  of  the  off-  line  facility. 


A brief  review  of  the  major  AWAVS  com- 
ponents is  necessary  to  provide  a background  for 
discussion.  The  AWAVS  CIG  structure,  shown  in 
Figure  1,  is  comprised  or  three  (3)  major  systems: 
an  on-line  system,  an  off-line  data  base  creation 
facility,  and  a non-real-time  camera  station. 

The  on-line  system  shown  in  Figure  2 con- 
tains a DEC  PDP-11/55  GP  computer  with  card, 
tape,  disc  and  line  printer  peripherals,  a special- 
purpose  CIG  processor  built  by  General  Electric, 
and  an  Instructor's  Console  with  TV  monitors  and 
joystick  control.  Special  features,  such  as  raster 
mapping  for  the  wide-angle  dome  display,  multiple 
resolution  rates  (525,  825,  or  1023  raster  lines), 
dynamic  update  of  on-line  storage,  face  level  of 
detail  blending  controls,  independent  viewpoint 
control,  and  collision  detection  provide  a multiple 
application  research  capacity  never  before  incor- 
porated in  a single  CIG  system.  These  features 
and  the  basic  on-line  architecture  comprised  of 
Frame  I,  Frame  n,  and  Frame  HI  organizational 
structures  similar  to  other  CIG  systems  covered 
in  papers  referenced  herein  will  not  be  discussed 
further  in  this  paper. 

The  off-line  data  base  creation  facility  shown 
in  Figjre  3 contains  a digitizing  table,  an  Applicon 
ealigraphic  storage  tube  display  and  software  sys- 
tem, and  a VAX-11/780  computer.  It  is  a com- 
pletely standalone  facility  that  allows  entire  data 
bases  to  be  digitized,  viewed  and  corrected  in 
ealigraphic  display,  and  formatted  for  real-time 
operation  with  no  on-line  system  time  required. 

Also  shown  in  Figure  3 is  the  AWAVS  camera 
station  consisting  of  a Dicomed  recording  monitor 
with  still  and  moving  film  capabilities  connected  to 
the  VAX  11/780  computer.  K is  operated  in  non- 
real-time  from  a software  program  that  simulates 
the  complete  on-line  system.  The  camera  station 
allows  various  algorithm  changes  to  be  imple- 
mented and  studied  without  the  necessity  of  hard- 
ware changes.  It  is  a general  research  tool  that 
allows  non-real-time  generation  of  still  photo- 
graphs and  motion  film  clips  of  high-detail 
visual  environments  far  exceeding  the  capacity  of 
any  existing  real-time  hardware.  "What  IF’ 
questions  about  data  base  content  and  image  gener- 
ator requirements  can  be  answered  without 
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hardware  prototypes.  Operational  characteristics 
for  image  generator  specifications  of  future  train- 
ing devices  can  be  determined  by  project  team 
personnel  well  in  advance  of  prototype  development. 
This  type  of  capability  was  never  required  of  past 
CIG  systems  designated  for  training  installations. 

DKVELOPING  A DATA  BASE 

In  any  CIG  system,  the  ultimate  product  and 
Anal  subjective  measure  of  the  system  is  the  data 
base  and  the  realism  of  its  visual  offspring.  This 
being  the  case,  the  system  tends  to  be  judged 
largely  by  the  effectiveness  of  its  data  base.  Con- 
sequently, the  data  base  becomes  the  instrument 
of  compromise.  In  the  final  development , it  must 
be  bent,  shaped,  and  manipulated  to  produce  the 
most  pleasing  dynamic  scene  while  being  con- 
strained by  and  compensating  for  whatever  are  the 
particular  system  hardware  and  software  restric- 
tions. 

An  off-line  data  base  creation  facility 
(functionally  shown  in  Figure  4)  will  improve  on 
the  time  and  effort  required  to  produce  the  first 
pass  data  base  through  use  of  interactive  graphic 
software  and  library  retrieval  of  stored  proto- 
objects. File  management  facilities  allow  basic 
object  definition  to  be  accumulated  on  mass  stor- 
age, then  retrieved  for  direct  utilization  or  modi- 
fication in  a visual  environment.  However,  the 
result  must  be  critiqued  on  the  actual  display 
monitors  in  order  to  determine  the  final  necessary 
modifications  and  enhancement.  Off-line  facilities 
with  some  type  of  visual  feedback  (Applicon  cali- 
graphics  for  AWAVS)  can  eliminate  some  of  the 
more  obvious  errors,  but  the  final  judgement  must 
be  done  in  a real-time  dynamic  movement  display 
with  true  perspective  and  correct  system  color 
characteristics. 

The  net  result  is  that  development  of  a new 
data  base  takes  on  an  organic  aspect  starting  with 
the  original  digitized  kernel  and  being  pruned  and 
culled  during  the  necessary  iterations  of  viewing 
the  results  on-line,  modifying  the  data  off-line, 
and  then  reviewing  the  modifications  on  the  on- 
line system. 

Since  the  final  raster  video  observed  by  the 
designer,  in  general,  does  not  contain  any  infor- 
mation to  locate  a particular  object  or  face  from 
its  off-line  source  (and  as  data  bases  become  ever 
larger,  the  connection  becomes  more  tenuous), 
the  designer  is  reduced  to  viewing  a scene  and 
making  notes  as  to  which  model/object/face 
requires  modification  based  on  personal  knowledge 


of  how  that  data  base  is  constructed.  The  designer 
must  then  return  to  the  off-line  facility  to  locate 
the  proper  card  deck,  disc  file  etc. , make  the 
assumed  changes  and  review  the  scene  to  deter- 
mine the  results.  This  is  a feasible  undertaking 
for  specialized  data  bases  or  easily  Identified 
models,  but  becomes  almost  an  impossibility  for 
misplaced  vertices,  lights,  objects  or  generally 
nondescript  terrain. 

ADDING  INTERACTIVE  CONTROLS 

In  order  to  maximize  the  contribution  of  an 
off-line  creation  facility,  it  is  necessary  to  design 
the  special  processor  hardware  with  a "tracing" 
ability  in  order  to  locate  the  off-line  source  of 
displayed  raster  images.  Without  this  capability, 
the  bulk  of  data  base  preparation  effort  stays  in  the 
"craftsman"  category  requiring  excessive  itera- 
tions during  the  real-time  viewing-modification 
cycle  and  undermining  the  advantages  of  an  off- 
line data  base  creation  facility. 

This  tracing  ability  of  the  hardware  must 
be  designed  to  provide  all  of  the  hooks  and  handles 
that  are  necessary  to  completely  identify  and 
locate  each  specific  item  of  an  environment.  The 
structure  of  each  system  will  dictate  the  unique 
hooks  and  handles  required  to  trace  the  format  of 
model/object/face  groupings  and  associated  ad- 
dressing schemes. 

The  AWAVS  system  has  data  bases  orga- 
nized into  a block/model  hierachy  (Figure  5)  with 
a given  data  block  containing  from  1 to  128  models 
and  each  model  capable  of  being  modeled  for  8 
separate  levels  of  detail.  Each  level  of  detail  is 
an  organization  of  objects,  each  object  is  a set  of 
faces,  and  each  face  a set  of  vertices.  The  entire 
environment  is  processed  by  the  special-purpose 
processor  hardware  using  a pointer  system  with 
all  pointers  relative  to  the  entire  data  block  start- 
ing at  address  zero.  Each  data  block  is  assigned 
by  the  on-line  computer  a starting  address  which 
is  added  as  an  offset  to  all  subsequent  pointers  of 
that  data  block. 

In  order  to  relate  raster  information  back 
into  this  data  structure,  it  is  first  necessary  to 
produce  a visible  raster  mark  on  the  display. 

This  is  done  on  AWAVS,  shown  in  Figure  6,  by 
Inserting  a pointer  into  the  video  processing 
portion  (Frame  III)  of  the  hardware  with  computer- 
controlled  I,J  values;  I being  the  raster  line  value 
and  the  J the  element  value.  This  marker  has  a 
dedicated  color  number,  so  it  can  be  changed  as 
needed  (white  for  night  scenes  for  instance)  and 
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always  has  priority  over  all  faces  in  the  scene. 

The  pointer  is  thus  moveable  under  operator  con- 
trol and  can  be  placed  over  any  face  in  a given 
scene. 

The  hardware  face  number  of  the  face  being 
"touched"  by  the  pointer  is  captured  by  special 
hardware  activated  by  the  pointer  controls.  This 
hardware  face  number  can  then  be  fed  to  the  front- 
end  hardware  (Frame  II)  that  is  processing  the  on- 
line environment  data  blocks.  There  is  special 
search  hardware  built  into  the  Frame  II  section 
that  uses  this  hardware  face  number  as  a key  and 
through  several  raster  frames  of  processing 
captures  first  the  object  environment  face  number, 
then  the  active  model  number,  then  the  environ- 
ment model  number  and  level  of  detail,  and  finally 
the  data  block  from  which  that  particular  face 
originated.  Each  specific  number  is  a reference 
provided  during  the  off-line  construction  of  that 
data  block  and  the  captured  set  is  sufficient  to 
uniquely  identify  an  object.  All  captured  infor- 
mation is  held  in  data  registers  that  are  accessible 
to  the  on-line  computer. 

Since  point  lights,  unlike  edges,  are  not 
generally  part  of  a visible  face,  it  was  necessary 
to  design  a point  light  lasso  feature.  In  this  mode 
of  operation  whenever  the  end  of  the  pointer  is 
within  a 4 x 4 pixel  range  of  a point  light,  that 
particular  point  light  is  captured  bv  special  hard- 
ware that  will  trigger  the  process  described 
previously  for  edge  faces. 

SPECIAL  INTERACTIVE  SOFTWARE 

At  this  point,  with  the  block,  model  and  face 
locations  captured,  the  guess  work  has  been  taken 
out  of  modifying  a specific  feature  and  a skilled 
operator  could  make  any  desired  changes,  but  still 
in  an  off-line  mode  requiring  further  review  and 
appraisal  of  the  results.  For  the  AWAVS  system, 
however,  a special  interactive  software  package 
was  developed  to  close  the  loop  in  data  base  modi- 
fications. With  this  software  package,  which  is 
capable  of  "flying"  the  pointer  using  the  debug  joy- 
stick, it  is  possible  to  not  only  collect  the  infor- 
mation but  to  then  edit  and  add  to  the  data  base 
under  review,  and  see  the  results  in  real-time  as 
indicated  in  Figure  7.  By  temporarily  changing 
vertices  in  the  Frame  II  data  storage,  edges  can 
be  moved  and  point  lights  relocated.  Face  colors 
can  be  modified,  levels  of  details  deleted,  added 
or  adjusted  and  general  environment  editing 
accomplished  in  operator  real-time  mode. 

More  extensive  data  base  modifications, 
such  as  translating  and  rotating  entire  models  are 


accomplished  by  temporarily  assigning  the  indi- 
cated model  to  a moving  model  coordinate  system. 
The  model  can  then  be  flown  under  joystick  or 
terminal  control  to  a new  position  and  attitude  in 
the  environment.  If  lights,  edges,  or  faces  are  to 
be  created  to  be  added  to  the  environment,  the 
affected  model  will  be  transferred  to  an  unused 
section  of  memory  and  pointers  adjusted  accord- 
ingly in  such  a manner  as  to  provide  the  additional 
memory  slots  necessary  to  contain  the  newly 
created  data. 

The  operator  interfaces  with  the  interactive 
software  using  simple  edit  commands  and  receiv- 
ing prompting  and  requested  information  (vertex 
locations,  colors,  etc.)  from  the  program.  He 
also  receives  visual  cues  on  the  screen  such  as 
faces  touched  by  the  pointer  will  flash  and  captured 
point  lights  will  blink  in  addition  to  the  inestimable 
information  gained  by  seeing  the  immediate  result 
of  a modification.  If  a particular  modification  does 
not  produce  desired  results,  it  can  be  returned  to 
the  original  state  or  otherwise  further  adjusted. 

Once  a series  of  modifications  has  achieved 
the  results  desired  by  the  operator,  he  can  termi- 
nate the  interac  process  and  request  a change 
summary.  This  will  result  in  providing  a step-by- 
step  listing  of  all  changes  that  were  produced 
during  the  data  base  review  session. 

Since  the  on-line  changes  were  temporary, 
this  change  listing  is  then  input  to  the  off-line 
facility  to  create  the  final  repacked  data  base. 

This  procedure  is  necessary  for  two  reasons.  One 
is  to  maintain  configuration  control  of  the  data 
bases  via  the  off-line  facility.  The  other  is  to 
eliminate  from  the  on-line  interactive  software 
those  data  tasks,  such  as  data  packing  and  priority 
separation  plane  computation,  that  are  not  neces- 
sary for  quick  visual  checks  of  the  modifications, 
but  are  too  time  consuming  to  allow  adequate  man- 
machine  response  times. 

ILLUSTRATION 

Figure  8 shows  a scene  from  a data  base 
under  development.  The  modeler  has  activated  the 
pointer  and  is  using  it  to  identify  a face  to  be 
changed.  The  interactive  program  provides  the 
modeler  with  a printout  identifying  this  as  face 
#173  of  model  #1,  first  level  of  detail,  environ- 
ment data  block  #4.  The  modeler  could  request 
additional  information  if  desired  such  as,  the 
vertices  defining  the  face,  color  code  of  face,  etc. 
The  modeler  wishes  to  develop  a low-detail 
version  of  this  model  so  types  in  a deletion  com- 
mand which  will  cause  the  program  to  restructure 
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the  data  to  inhibit  processing  of  that  face.  Figure 
9 shows  the  changed  scene,  validating  in  an  imme- 
diate interactive  manner  that  the  desired  result 
was  achieved. 

SUMMARY 

The  result  of  the  overall  A WAVS  data  base 
creation  package  is  to  provide  a complete  set  of 
tools  for  the  designer  such  that  he  can  quickly 
create  and  digitize  a new  data  base  on  the  off-line 
facility,  completing  a modicum  of  first  level  debug 
with  the  cal  (graphic  display.  He  can  then  review 
and  modify  the  data  base  on  the  on-line  system  in 
real-time  with  a high  degree  of  confidence  that  the 
final  off-line  product  will  produce  the  intended 
results.  The  interactive  software  package  reduces 


the  on-line  system  time  required  by  increasing  the 
ability  to  identify  objects  to  be  changed  and  signifi- 
cantly reducing  the  number  of  iterations  necessary 
to  achieve  the  final  result. 
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Figure  3.  Off-Line  Facility  and  Camera  Station 


Figure  4.  Data  Base  Creation  Functional  Diagram 


Figure  8.  AWAVS  Data  Base  Being  Developed  — Pointer  Activated 


Figure  9.  Result  of  Modification  is  Immediately  Validated 
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VISUAL  ILLUSIONS  IN  FLIGHT 


ABSTRACT 

Both  the  effectiveness  of  pilot  training 
and  the  safety  of  flight  can  be  influenced 
by  the  distribution  of  texture  in  the  visual 
scene,  the  distance  to  which  the  eyes  accom- 
modate, and  the  associated  shifts  in  the 
apparent  size  and  distance  of  objects  in 
central  and  peripheral  vision.  Results  to 
date  indicate  that  these  factors  are  in- 
volved in  various  misjudgments  and  illusions 
experienced  by  pilots:  (1)  when  searching 
for  other  airborne  traffic  or  targets,  (2) 
when  making  approaches  to  airports  over  water 
at  night,  (S)  when  breaking  out  of  low 
clouds  on  a final  approach  to  a landing  by 
reference  to  head-up  or  head-down  displays, 
and  (4)  when  practicing  simulated  approaches 
and  landings  or  air-to-surface  weapon  deliv- 
eries by  reference  to  synthetically  generated 
visual  systems. 


WHAT  THIS  TALK  IS  ABOUT 

1.  Making  landing  approaches  over 
water  on  a dark  night  toward  a 
brightly  lighted  city. 

2.  Looking  for  intruding  airplanes 
from  the  flight  engineer's  seat. 

3.  Sitting  inside  a screened  porch 
and  trying  to  read  a NO  FISHING 
sign  down  by  the  lake. 

4.  Projecting  afterimages  onto  the 
walls  of  a football  stadium. 

5.  Watching  the  moon  rise  over  Miami. 

What  do  these  seemingly  unrelated  ac- 
tivities have  in  common?  And  what  does  all 
this  have  to  do  with  head-up  flight  displays, 
head-down  imaging  displays,  helmet-mounted 
displays,  and  visual  systems  for  contact 
flight  simulators?  In  each  case  visual  illu- 
sions occur:  systematic  misjudgments  of 
size  and  distance  relationships,  departures 
by  varying  amounts  from  the  so-called 
"size-distance  invariance  hypothesis." 


When  pilots  make  approaches  and  land- 
ings with  any  type  of  imaging  flight  dis- 
play projected  at  unity  magnification,  they 
tend  to  come  in  fast  and  long,  round  out 
high,  and  touch  down  hard.  On  the  final 
approach  the  runway  appears  smaller,  farther 
away,  and  higher  in  the  visual  field  than  it 
does  when  viewed  directly  from  the  same 
flight  path  on  a clear  day.  This  finding 
has  been  obtained  independently  both  with 
flight  periscopes  and  with  simulated  contact 
visual  systems  (Roscoe,  Hasler,  and  Dougher- 
ty, 1966;  Palmer  and  Cronn,  1975;  Roscoe, 
1976)  . 

In  stark  and  tragic  contrast,  when 
pilots  make  approaches  to  landings  over  water 
on  a dark  night  toward  a brightly  lighted 
city,  the  runway  appears  larger,  nearer,  and 
lower  in  the  visual  field  than  it  does  when 
viewed  directly  from  the  same  flight  path  on 
a clear  day.  On  several  occasions  in  recent 
years,  a commercial  airliner  has  landed  in 
the  water  short  of  the  airport  when  making  an 
approach  at  night.  Kraft  (1968;  1978)  has 
shown  that  pilots  will  systematically  mis- 
judge the  height  and  "tilt"  of  the  runway 
and  make  low  approaches  under  these  condi- 
tions. 

In  another  experiment  by  Kraft,  Farrell, 
and  Boucek  (1970) , a group  of  pilots  judged 
the  threat  of  midair  collision  with  intrud- 
ing airplanes  at  varying  distances  and  an- 
gles, none  of  which  represented  an  actual 
collision  threat.  The  pilots  were  presented 
a series  of  pictures  projected  onto  a screen 
viewed  from  a mocked-up  Boeing  737  cab.  When 
the  judgments  were  made  from  the  flight  en- 
gineer's seat,  as  opposed  to  the  pilot's 
seat,  the  same  pilots  consistently  judged 
the  intruders  to  be  a greater  threat  at  all 
ranges  out  to  3500  feet.  From  the  rear 
seat,  the  intruders  appeared  reliably  larger 
and  closer  than  from  the  front  seat. 

The  viewpoint  from  the  flight  engineer's 
seat  is  nearly  two  meters  from  the  windshield 
aperture;  from  the  pilot's  seat  it  is  less 
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than  one  meter.  Furthermore,  the  view  from 
the  flight  engineer's  seat  includes  much  of 
the  instrument  panel  when  searching  for  in- 
truders. When  searching  head-up  from  the 
pilot's  seat,  the  instrument  panel  appears 
in  the  dim  periphery;  the  pilot  sees  mainly 
empty  space  through  a windshield  that  re- 
flects glare  and  may  be  dirty  or  scratched. 
These  conditions  suggest  that  pilots  may  un- 
knowingly be  subject  to  the  "Mandelbaum 
Effect." 

In  1960,  Mandelbaum  reported  an  infor- 
mal experiment  in  which  he  asked  subjects  to 
read  a distant  sign  from  a screen-enclosed 
porch.  For  each  observer  he  found  a criti- 
cal distance  from  the  screen  at  which  the 
sign  could  not  be  read,  although  it  was 
clearly  legible  from  other  distances,  either 
nearer  or  farther.  Upon  questioning,  the 
subjects  realized  that  they  could  not  help 
focusing  on  the  screen  from  the  critical  dis- 
tance but  could  readily  focus  on  the  sign  by 
moving  either  nearer  or  farther  from  the 
screen  or  by  quick  movements  of  the  head  from 
side  to  side.  Mandelbaum  concluded  that  the 
"effect"  was  due  to  involuntary  accommoua- 
tion. 

It  was  noted  that  the  critical  distance 
from  the  screen  varied  from  person  to  person, 
with  an  average  distance  of  about  one  meter. 
In  an  ingenious  series  of  experiments  at 
Pennsylvania  State  University,  Owens  (in 
press)  has  subsequently  determined  that  the 
critical  distance  is  the  distance  of  the  in- 
dividual's dark  focus,  or  resting  accommoda- 
tion. For  the  young,  healthy  eyeball,  that 
distance  on  average  is  slightly  less  than 
one  meter  (slightly  more  than  one  diopter  in 
optical  terms),  the  distance  of  the  dirty 
windshield  from  the  pilot.  Almost  any  tex- 
tured visual  stimulus  at  that  distance  is  a 
powerful  involuntary  "accommodation  trap." 

A SCIENTIFIC  MYSTERY 

In  addition  to  the  misjudgments  of  size 
and  distance  discussed  so  far,  bias  errors 
in  depth  discrimination  have  been  discovered 
independently  by  designers  of  submarine  peri- 
scopes, tank  periscopes,  laboratory  micro- 
scopes, "one-power"  scopes  for  shotguns,  and 
helmet-mounted  CRT  displays.  All  require 
some  optical  magnification  to  cause  objects 
to  appear  at  the  same  distances  as  when 
viewed  by  the  naked  eye.  Furthermore , all  in- 
volve reductions  in  the  field  of  view  and  in 
the  textural  gradient  that  serves  as  the 
stimulus  for  distant  accommodation.  These 
biased  perceptions  of  size  and  distance  are 
not  fully  explained,  at  least  not  sufficient- 
ly to  give  comfort  to  the  pilots  and  passen- 
gers of  airplanes. 

The  mystery  manifests  itself  in  many 
forms  that  have  puzzled  psychologists  from 


Ptolemy  who  tried  to  explain  the  "moon  illu- 
sion" to  Young  (1952)  who  had  subjects  pro- 
ject^ visual  afterimages  onto  the  walls  of 
the  Ohio  State  football  stadium  from  various 
distances  across  an  open  field.  The  farther 
the  afterimage  is  projected,  the  larger  it 
appears,  but  not  in  direct  proportion  as 
would  be  predicted  by  the  size-distance  in- 
variance hypothesis.  The  "size"  of  the  moon 
also  varies  with  the  extent  of  the  visible 
textural  gradient,  appearing  larger  over  a 
distant  horizon  than  it  does  over  a near 
horizon,  as  shown  by  Kaufman  and  Rock  (1962). 

Throughout  the  literature  of  vision  re- 
search may  be  found  additional  examples  of 
unexplained  experimental  findings  and  assort- 
ed "optical  illusions"  that  may  be  related  to 
the  observations  by  Wheatstone  (1852)  and 
Helmholtz  (1867/1962),  and  more  recently 
verified  experimentally  by  Biersdorf  and 
Baird  (1966),  by  Leibowitz,  Shiina,  and 
Hennessy  (1972),  and  by  Roscoe,  Olzak,  and 
Randle  (1976),  that  the  apparent  size  of  an 
object  changes  with  shifts  in  the  distance 
to  which  the  eye  is  accommodated.  The  phe- 
nomenon can  be  illustrated  by  any  one  of 
several  simple  experiments. 

For  example:  close  one  eye,  focus  your 
open  eye  on  your  thumb  held  at  arm's  length, 
observe  a more  distant  object  such  as  a window 
or  a picture  on  the  wall,  and  while  continuing 
to  focus  on  your  thumb,  draw  it  toward  you  and 
observe  the  change  in  the  size  of  thq  window 
or  picture.  Better  yet:  look  at  the  moon 
through  a peekhole  through  your  fist,  alter- 
nately closing  and  opening  the  other  eye.  Not 
only  can  the  moon  on  the  horizon  be  made  to 
appear  smaller,  but  also  the  moon  overhead  can 
be  made  to  vary  in  apparent  size  by  a surpris- 
ing amount . 

INVESTIGATING  THE  MYSTERY 

To  investigate  the  possibility  that 
shifts  in  apparent  size  are  associated  with 
shifts  in  visual  accommodation  distance,  an 
experiment  was  conducted  at  NASA's  Ames  Re- 
search Center  in  which  visual  accommodation 
was  measured  continuously,  using  a Crane- 
Cornsweet  infrared  optometer,  while  subjects 
viewed  discs  that  subtended  a constant  3° 
angle  at  distances  ranging  from  1/4  to  4 
meters,  with  and  without  the  distance  cues 
provided  by  a sometimes  visible  textural 
gradient  (Roscoe,  et  al.,  1976).  Shifts  from 
binocular  to  monocular  viewing  were  accompan- 
ied by  shifts  in  accommodation,  both  inward 
and  outward,  toward  an  intermediate  distance 
of  a little  less  than  one  meter  (1.13  diopters, 
on  average) . 

The  reliable  inward  shifts  from  the  most 
distant  targets  at  4 meters  were  accompanied 
by  reliable  reductions  in  apparent  size.  A 
contingency  analysis,  summarized  in  Table  1, 


TABLE  1. 


SUMMARY  OF  CONTINGENT  PROBABILITY  ANALYSIS 
OF  PREDICTED  JUDGMENTS  OF  RELATIVE  SIZE  WITH 
CORRESPONDING  SHIFTS  IN  ACCOMMODATION 
(CHANCE  PROBABILITY  OF  CONTINGENCY  = 0.25). 

DISTANCE  TO  TARGET.  METERS 


i 

1-1/2 

2 

4 

CHANCE  CONTINGENCY 

.25 

.25 

.25 

.25 

OBSERVED  CONTINGENCY 

.23 

.36 

.38 

.45 

2 

Chi 



7.59 

11.34 

25.01 

P 

n.  r . 

<•01 

<.002 

<.001 

showed  that  the  correlation  between  shifts  in 
apparent  size  and  shifts  toward  the  resting 
accommodation  distances  of  the  individual  sub- 
jects increased  with  target  distance.  At  one 
meter  there  was  a chance  relationship,  at 
1 1/2  meters  the  contingency  was  reliable  at 
the  £ <.01  level,  at  two  and  four  meters  the 

£ values  were  < .002  and  < .001,  respectively. 
At  four  meters  the  contingency  was  almost  2 
to  1 greater  than  chance,  which  shows  a highly 
likely  relationship  but  still  leaves  a lot  of 
variance  unaccounted  for. 


To  clarify  the  relationship  between 
accommodation  and  apparent  size,  12  of  the 
original  16  subjects  were  tested  on  near 
(1/4-meter)  and  far  (4-meter)  targets  with 
a 1-mm  diameter  artificial  pupil  placed  8 cm 
from  the  entrance  plane  of  the  eye  used  for 
monocular  viewing.  An  artificial  pupil  al- 
lows the  eye  to  lapse  farther  toward  its 
resting  position  without  causing  a blurred 
image  (Hennessy  and  Leibowitz,  1975).  In 
binocular  viewing  the  second  eye  was  unob- 
structed, thereby  requiring  more  accurate 
accommodation  to  obtain  a clear  image  of  the 
target.  The  results  of  this  comparison  are 
shown  in  Table  2. 

The  arrows  in  Table  2 indicate  the 
shifts  in  accommodation  toward  the  resting 
position  from  binocular  to  monocular  view- 
ing, and  the  plus-signs  indicate  coincidence 
of  positive  accommodation  shifts  and  "Monoc- 
ular Smaller"  judgments,  or  conversely, 
negative  accommodation  shifts  and  "Monocular 
Larger"  judgments.  The  introduction  of  the 
artificial  pupil  clarifies  the  relationship: 
for  the  4-meter  target,  the  coincidence  is 
virtually  perfect,  23  of  24  cases  in  agree- 
ment; for  the  1/4-meter  target,  accommoda- 
tion shifted  in  the  predicted  direction  9 
times  in  12  under  both  light  and  dark  ambient 
illumination,  but  only  in  the  dark  is  there 
evidence  of  a trend  toward  "Monocular  Larger" 


TABLE  2. 

SHIFTS  IN  MEASURED  VISUAL  ACCOMMODATION  AND  JUDGMENTS  OF  THE  RELATIVE  SIZE 
OF  THREE-DEGREE  DISCS,  VIEWED  M0N0CULARLY  (M)  AND  BINOCULARLY  (B) 

AT  DISTANCES  OF  25  CM  (4.00  DIOPTERS)  AND  4 M (0.25  DIOPTER) 

UNDER  NORMAL  ROOM  LIGHTING  (LIGHT)  AND  REDUCED  ILLUMINATION  (DARK) , 
WITH  AN  ARTIFICIAL  PUPIL  IN  FRONT  OF  THE  LEFT  (MONOCULAR)  EYE. 

Distance  to  Target  Disc 

25  cm  (4.00  diopters) 4 m (0,25  diopter) 


Dark  Light  Light  Dark 


S 

B 

M 

B 

M 

M 

B 

M 

B 

1 

2.64 

-* 

2.07 

+ 

3.07 

-4 

2.27 

+ 

0.69 

■e 

0.24 

+ 

1.18 

♦ 0.28 

+ 

2 

3.70 

-► 

2.81 

+ 

3.88 

-► 

2.50 

1.06 

4- 

0.32 

+ 

0.12 

♦ -0.43 

3 

3.86 

2.86 

+ 

4.42 

-4 

1.78 

+ 

0.87 

4- 

0.26 

+ 

-0.21 

* -0.78 

+ 

4 

0.26 

-► 

0.17 

0.49 

0.79 

-0.15 

4“ 

-0.67 

+ 

-0.58 

♦ -0.97 

+ 

5 

1.86 

-► 

1.51 

2.18 

-4 

1.06 

-0.12 

4- 

-0.61 

+ 

0.17 

♦ -0.33 

6 

4.13 

-* 

2.86 

4.40 

-4 

3.38 

0.07 

4- 

-0.14 

+ 

0.53 

-1-  -0.56 

+ 

7 

3.04 

-► 

1.76 

+ 

3.75 

-4 

2.14 

1.02 

0.68 

+ 

0.63 

*■  0.39 

+ 

8 

4.30 

2.66 

+ 

4.66 

-4 

4.12 

+ 

0.26 

-0.11 

+ 

-0.10 

■*-  -0.54 

+ 

9 

2.18 

-4 

1.83 

+ 

1.71 

-4 

1.07 

+ 

-0.13 

4- 

-1.02 

+ 

-0.08 

-0.84 

+ 

10 

3.13 

-*• 

1.94 

+ 

3.95 

3.15 

+ 

0.58 

4- 

0.06 

+ 

0.22 

- 0.02 

4- 

11 
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-0.33 

•*-  -0.41 

+ 

Mean 

2.92 

2.06 

3.23 

2.19 

0.51 

-0.05 

0.23 

-0.31 

Legend:  Arrow  indicates  that  shift  from  binocular  to  monocular  accommodation 

is  toward  intermediate  distance.  + indicates  that  a positive  shift  in 
accommodation  is  accompanied  by  a judgment  of  "Monocular  Smaller"  or, 
conversely,  a negative  shift  by  "Monocular  Larger." 
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judgments  with  outward  shifts  in  accommoda- 
tion (9  of  12  cases,  £ < .10). 

In  addition  to  the  fact  that  correla- 
tions do  not  guarantee  causal  relationships, 
these  findings  are  equivocal  because  of  the 
confounding  of  shifts  in  accommodation,  which 
were  measured,  with  shifts  in  convergence  be- 
tween binocular  and  monocular  viewing,  which 
were  not  measured.  Furthermore,  the  accommo- 
dation data  are  not  sufficiently  clean  for 
comfort,  and  a few  individual  data  are  sus- 
pect by  inspection.  Nevertheless,  neither 
the  data  nor  their  implications  can  be  dis- 
counted as  completely  spurious  in  the  absence 
of  better  data.  In  any  case,  the  mystery  is 
not  so  much  how  we  judge  the  size  and  dis- 
tance of  near  objects  that  afford  binocular 
cues  as  it  is  how  we  judge  distant  objects 
that  provide  only  monocular  cues. 

To  gain  a better  understanding  of  the 
effects  of  visual  accommodation  upon  judg- 
ments in  tasks  involving  complex,  dynamic 
visual  scenes,  another  experiment  was  recent- 
ly conducted  at  Ames  Research  Center  using 
the  Crane-Cornsweet  infrared  optometer  and 
an  experimental  night-landing  visual  display 
generated  by  a digital  computer  (Randle, 
Roscoe,  and  Petitt,  in  press).  Professional 
pilots  made  judgments  of  whether  they  would 
undershoot  or  overshoot  their  landing  aim- 
point  as  the  computer  flew  their  simulated 
jet  transport  on  final  approaches  to  the 
computer-generated  airport  scene. 

Experimental  variables  included:  (1) 
the  magnification  of  the  visual  scene,  which 
was  varied  in  five  steps  between  0.83  and 
1.67,  (2)  the  visual  accommodation  distance 

induced  by  five  sets  of  ophthalmic  lenses 
with  dioptric  powers  ranging  from  zero  to 
three,  (3)  the  actual  descent  path  of  the 
simulated  airplane,  which  included  overshoots 
and  undershoots  as  well  as  correct  landing 
approaches,  and  finally,  (4)  whether  the 
landing  scene  was  presented  as  a real  image 
viewed  directly  on  a TV  monitor  or  a virtual 
image  produced  by  a collimating  field  lens 
mounted  between  the  monitor  and  the  pilot. 

The  first  finding  was  that  the  eye  does 
not  respond  obediently  to  the  accommodation 
distances  called  for  by  ophthalmic  lenses; 
the  eye  is  lazy  and  reluctant  to  be  drawn 
away  from  its  intermediate  resting  position. 
The  brain,  in  turn,  seems  happy  to  accept  an 
amazingly  out-of-focus  image  uncritically 
and,  in  fact,  without  conscious  recognition 
that  it  is  out  of  focus.  In  response  to 
ophthalmic  lenses  covering  the  range  from 
zero  to  three  diopters,  the  pilots’  eyes,  on 
average,  accommodated  to  the  virtual  and 
real  images  over  ranges  of  only  1.27  and 
1.46  D,  respectively. 


Despite  the  relatively  small  shifts  in 
accommodation  "induced"  by  the  ophthalmic 
lenses,  there  were  statistically  reliable  in- 
teractions in  the  predicted  directions  be- 
tween actual  accommodation  levels  and  the 
pilots'  judgments  of  whether  they  would  over- 
shoot or  undershoot  their  landings.  There 
is  now  little  doubt  that  such  judgments  are 
related  in  some  complicated  way  to  visual 
accommodation  distance,  which,  in  turn,  is 
affected  far  more  by  the  various  viewing  con- 
ditions encountered  in  the  spectrum  of  nor- 
mal flight  operations  than  it  was  by  the  oph- 
thalmic lenses  used. 

An  experiment  typically  raises  more 
questions  than  it  answers,  and  this  one  was 
no  exception.  The  pilots  made  two  judgments 
along  the  final  approach,  the  first  at  20 
seconds,  or  4000  feet,  before  passing  the 
runway  aimpoint  and  the  second  at  10  seconds, 
or  2000  feet.  With  unity  image  magnifica- 
tion, they  predominantly  indicated  an  over- 
shoot on  the  first  judgment  and  an  undershoot 
on  the  second.  If  they  had  been  flying  man- 
ually, they  would  have  tended  to  overshoot. 
Veridical  judgments  were  obtained  at  the 
nearer  distance  with  an  image  magnification 
of  1.25,  as  has  been  found  with  flight  peri- 
scopes (Roscoe,  et  al.,  1966). 

The  possible  explanations  for  this  cu- 
rious reversal  in  judgments  are  infinite.  Of 
course,  the  finding  might  be  unique  to  the 
particular  computer-generated  night  visual 
scene  used.  However,  based  in  Kraft's  find- 
ings , it  could  be  that  pilots  habitually 
make  low  approaches  at  night  to  avoid  over- 
shooting and,  when  they  are  still  4000  feet 
out,  "expect"  the  runway  to  appear  as  it  does 
from  a position  below  the  3-degree  approach 
path.  At  2000  feet  out,  they  can  see  their 
position  better  and  maintain  thrust  to  carry 
them  to  the  touchdown. 

At  4000  feet  out  the  dominant  cues  for 
accommodation,  namely,  the  airport  lighting 
system  and  the  lighted  city  beyond,  appear  as 
a thin  horizontal  band  of  point  sources  at  a 
relatively  great  distance;  far  accommodation 
is  required  to  resolve  the  scene.  As  the  air- 
plane approaches  the  runway,  the  band  deepens 
and  comes  nearer;  the  runway  lights  are  more 
easily  resolved,  and  accommodation  drifts  in- 
ward from  its  distant  "trap."  The  so-called 
"size  constancy"  of  the  runway  is  not  main- 
tained; in  effect  it  shrinks  a little,  and 
pilots  tend  to  overshoot  their  aimpoint  once 
they  have  safely  crossed  the  threshold. 

To  test  this  wild  speculation,  two  ex- 
periments have  just  been  conducted  at  the 
University  of  Illinois  (Iavecchia,  1978).  A 
1/2-degree  collimated  disc  of  light,  simulat- 
ing the  moon,  was  projected  onto  a 45-degree 
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combining  glass  so  that  it  appeared  as  a 
virtual  image  superposed  on  the  outside  visu- 
al scene  (a  la  Kaufman  and  Rock).  A second, 
comparison  "moon"  of  adjustable  diameter  was 
presented  as  a real  image  at  a distance  of 
one  meter  in  an  otherwise  dark  surrounding.  The 
two  views  were  presented  alternately  in  the 
same  visual  position  by  means  of  a sliding 
mirror  arrangement,  and  the  subject  adjusted 
the  diameter  of  the  comparison  until  a satis- 
factory apparent-size  match  was  obtained. 

In  the  first  experiment,  conducted  in 
clear  daylight,  subjects  viewed  the  colli- 
mated moon  against  the  scene  visible  from 
corresponding  windows  of  the  third  to  the 
eighth  floors  of  the  Psychology  Building 
overlooking  the  Urbana  campus  and  residential 
area.  On  the  third  floor  the  moon  was  pro- 
jected against  the  roof  of  a nearby  sorority 
house,  and  on  successively  higher  floors 
against  successively  more  distant  rooftops 
and  large  trees.  At  the  fifth  and  sixth 
floors  it  appeared  just  above  the  horizon, 
and  on  the  seventh  and  eighth,  higher  and 
higher  above  the  horizon.  The  apparent  size 
of  the  moon  increased  from  the  third  to  the 
sixth  floors  and  then  reversed  itself  as  it 
rose  above  the  horizon. 

The  mean  apparent  size  ratios  of  the 
moon,  relative  to  its  apparent  size  when  pro- 
jected onto  a newspaper  viewed  from  one  meter, 
were  (3rd  floor)  1.143,  (4th)  1.250,  (5th) 
1.311,  (6th)  1.364,  (7th)  1.330,  (8th)  1.282. 
These  means  differed  reliably  (j)  < .05).  As 
the  moon  was  projected  against  increasingly 
distant  surfaces  from  the  3rd  through  the  6th 
floors,  its  apparent  size  increased  monotoni- 
cally.  From  the  6th  floor,  the  moon  was  pro- 
jected against  the  sky  just  above  the  most 
distant  surface  texture.  From  the  7th  and  8th 
floors,  it  was  projected  against  the  sky 
higher  and  higher  above  the  horizon. 

In  the  second  experiment  the  distance  and 
vertical  position  of  visible  texture  was  mani- 
pulated more  systematically  by  viewing  the 
scene  from  the  fifth  floor  through  a series  of 
masks.  Four  of  the  masks  revealed  horizontal 
bands  of  texture  in  the  Near,  Intermediate, 

Far,  and  Very  Far  visual  fields.  Another  mask 
obscured  all  surface  texture  in  the  visual 
field  so  that  the  moon  was  projected  against 
the  open  sky  just  above  the  "horizon"  formed 
by  the  mask.  Finally,  a clear  mask  revealed 
the  entire  scene.  The  results  of  these  tests 
clarify  the  situation. 

When  viewing  the  moon  against  the 
"unmasked"  background  scene  (clear-mask 
control  condition),  its  apparent  size  ratio 
was  1.369.  With  the  mask  that  revealed  only 
Hear  texture,  it  was  1.225;  for  Intermediate 
texture,  1.235;  for  Far  texture,  1.289;  and 
for  Very  Far  texture,  1.395.  With  the  mask 
that  obscured  all  surface  texture  below  the 


horizon  (similar  to  a view  of  the  moon  over- 
head), the  apparent  size  ratio  dropped  abruptly 
to  1.136,  only  slightly  larger  than  its 
apparent  size  when  projected  onto  the  news- 
paper viewed  from  a distance  of  one  meter. 

What  these  two  experiments  show  is  that 
the  apparent  size  of  objects  well  beyond  the 
6-meter,  or  20-foot,  distance  to  "optical 
infinity"  change  reliably  with  changes  in  the 
spatial  distribution  of  textural  stimuli  to 
accommodation  in  the  background  visual  scene. 
The  greater  the  distance  through  empty  space 
to  resolvable  texture,  the  larger  the  apparent 
size  of  centrally  fixated  objects,  such  as  the 
moon  or  an  airport  runway.  As  the  textural 
pattern  extends  downward  or  moves  nearer,  the 
central  object  fails  to  maintain  a constant 
"apparent  size."  As  the  pilot  approaches  a 
runway  over  water  at  night,  his  visual  image 
of  the  runway  grows,  but  not  in  perfectly 
inverse  proportion  to  distance  remaining. 

When  no  resolvable  background  texture  is 
present,  as  when  viewing  the  moon  against  a 
clear  sky,  the  textureless  moon  provides  an 
inadequate  stimulus  to  distant  accommodation 
and  shrinks  in  size,  as  do  the  symbols  of  a 
head-up  display  when  flying  in  clouds.  Even 
a partially  clouded  sky  apparently  cannot  hold 
distant  accommodation  to  a textureless  colli- 
mated moon  or  display  symbols.  Thus,  the 
"moon  illusion"  is  not  manifested  by  a spuri- 
ously large  moon  on  the  horizon  but  rather  by 
a perceptually  shrunken  moon  overhead. 

IMPLICATIONS  FOR  FLIGHT  SAFETY 

For  years  Kraft,  Hennessy,  and  several 
other  investigators  have  recommended  that 
pilots  routinely  wear  bifocal  lenses  at 
night  and  when  making  IFR  approaches  in  day- 
light conditions.  The  lower  section  would 
optimize  their  vision  for  instrument  panel 
and  chart  viewing  distances.  The  upper 
section  would  provide  negative  correction 
to  aid  distant  accommodation  for  outside 
viewing.  Owens  and  Leibowitz  (1976)  have 
shown  that,  if  night  drivers  with  normal 
vision  are  asked  to  select  the  lenses  that 
allow  them  to  see  best,  they  will  choose 
those  with  a negative  correction  halfway 
between  tbeir  dark  focus  and  optical  in- 
finity. 

To  combat  the  possible  underaccommoda- 
tion experienced  by  some  pilots  while  making 
"black  hole"  approaches  over  water  at  night, 
lead-in  light  buoys  should  be  considered 
and  tested  for  use  at  major  airports.  Al- 
though no  specific  data  are  available,  it 
would  be  expected  that,  in  the  absence  of 
visible  texture  in  the  near  field,  pilots 
with  extremely  distant  dark  focus  would  be 
the  ones  who  tend  to  make  low  approaches  at 
night  and  occasionally  land  in  the  ocean. 
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Perhaps  they  should  wear  positive  correc- 
tive lenses  at  night,  but  evidently  no  such 
tests  have  been  made. 

The  use  of  head-up  displays  for  night 
and  IFR  approaches  warrants  further  investi- 
gation. It  has  been  tacitly  assumed  and 
strongly  asserted  by  the  advocates  of  such 
displays  that  the  collimated  presentation 
prepares  the  eyes  to  resolve  immediately 
whatever  is  out  there  to  be  seen.  Avail- 
able experimental  evidence  does  not  support 
that  assertion.  The  CIG/NVS  landing 
approach  study  at  Ames  (Randle,  et  al.)  and 
the  moon-illusion  studies  at  Illinois 
(Iavecchia)  clearly  show  that  collimating 
bold,  well  defined  symbology,  whether 
viewed  directly  or  reflected  from  a combin- 
ing glass,  does  not  necessarily  call  the 
eyes  to  a far  accommodation  distance.  When 
the  pilot  breaks  out  of  the  clouds,  rapid 
negative  accommodation  is  required,  and 
the  scene  "explodes." 

IMPLICATIONS  FOR  PILOT 

SELECTION  AND  TRAINING 

The  evidence  presented  suggests  that 
dark  focus,  or  resting  accommodation  dis- 
tance, in  addition  to  basic  visual  acuity 
and  color  vision,  should  be  taken  into 
account  in  pilot  selection  and  assignment. 
Having  a far  resting  accommodation  distance 
might  be  one  basis  for  assigning  military 
pilots  to  air  combat  duty;  they  should  be 
less  troubled  by  empty-field  myopia.  Those 
with  a nearer  resting  position  might  bene- 
fit from  negative  lenses,  as  in  the  case  of 
civilian  pilots  watching  for  intruders.  As 
pilots  get  older  their  resting  accommoda- 
tion may  retreat  into  the  distance,  occa- 
sionally to  a point  at  which  they  could 
have  serious  problems  making  "black  hole" 
approaches . 


Office  of  Scientific  Research,  AFSC,  with 
Dr.  Alfred  R.  Fregly  as  Scientific  Monitor. 
Technical  review  of  the  experimental  plans 
for  the  AFOSR  studies  was  provided  by  scien- 
tists from  the  Aerospace  Medical  Research 
Laboratory,  Wright-Patterson  Air  Force  Base, 
Ohio . 

Oskar  F.  Richter,  Department  of  Psy- 
chology, University  of  Illinois  at  Urbana- 
Champaign,  designed  and  constructed  the 
apparatus  used  in  the  AFOSR  moon-illusion 
experiments.  Scientific  consultation  was 
provided  by  Dr.  Robert  T.  Hennessy  of  the 
Canyon  Research  Group,  by  Patrick  FitzHenry 
of  Uliana  Aviation  Sciences,  and  without 
recompense,  by  Robert  J.  Randle  of  the  Ames 
Research  Center.  Morris  H.  Maitland  prepared 
all  graphic  materials  for  the  Illinois  ex- 
periments and  for  this  report. 

The  experiments  at  the  Ames  Research 
Center  were  conducted  by  Lynn  A.  Olzak  and 
John  Petitt  of  the  Hughes  Aircraft  Company 
and  by  Donna  Miller  of  the  Ames  Research 
Center.  The  experiments  at  the  University 
of  Illinois  were  conducted  by  Graduate 
Research  Assistants  Joyce  H.  Iavecchia  and 
Helene  P.  Iavecchia.  Kathleen  J.  Beringer 
typed  the  manuscript. 
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PILOT  JUDGMENT:  TRAINING  AND  EVALUATION 


DR.  RICHARD  S.  JENSEN 
University  of  Illinois  at  Urbana-Champaign 


JUDGMENT  — perhaps  the  key  .to 
longevity  in  naval  aviation.  Judgment  is 
obviously  critical  in  the  inflight  regime 
where  multiple  decisions  must  be  made  in  a 
timely,  correct,  and  often  irreversible 
fashion.  Just  as  important,  and  perhaps 
too  often  overlooked,  is  the  importance  of 
good  judgment  on  the  ground.  Knowing  when 
not  to  fly,  planning  flights  carefully, 
and  realizing  personal  limitations  are 
examples  of  good  judgment  displayed  on  the 
ground  (Dunn,  1977). 

INTRODUCTION 

From  the  beginning  of  aviation  history 
pilots  have  been  expected  to  exercise  a 
considerable  amount  of  judgment  in  the  overall 
task  of  flying  an  airplane.  However,  in 
recent  years,  increasing  demands  in  our 
society  for  safety,  dependability,  economy, 
effectiveness,  and  reduced  energy  consumption 
have  increased  the  complexity  of  civil  and 
military  flying  operations  magnifying  the 
pressures  for  good  pilot  judgment.  Further- 
more, technological  advances  that  have  eased 
much  of  the  pilot's  burden  for  precise 
aircraft  control  have  not  greatly  eased  the 
pilot's  decision-making  workload.  In  many 
cases  these  advances  have  only  created  demands 
for  higher  levels  of  skill,  knowledge,  and 
judgment  to  which  few  pilots  have  been 
trained,  and  the  training  costs  to  prepare 
them  to  operate  effectively  in  the  changing 
system  are  becoming  prohibitive. 

Flying  has  developed  so  rapidly  that 
there  has  been  little  time  for  a serious  study 
of  what  flying  is  all  about,  particularly  in 
terms  of  how  pilots  think.  Many  changes  in 
regulations  are  expedients  designed  to  solve 
problems  that  have  already  developed.  Often 
solutions  to  existing  problems  create  new 
ones,  which  in  turn,  are  "solved"  by  new 
regulations.  The  problem  of  training  new 
pilots  and  retraining  current  pilots  to 
facilitate  the  implementation  of  new 
procedures  and  regulations  in  a mobile  but 
energy  limited  society  is  just  beginning  to  be 
recognized  (Roscoe,  1974). 


aviation  regulations,  procedures,  and  per- 
formance limitations,  the  safety  of  a given 
flight  also  depends  upon  a significant  amount 
of  evaluation  and  interpretation  of  existing 
conditions  by  the  pilot. 

For  example,  the  conditions  used  to 
develop  flight  performance  values  for  a 
particular  type  of  airplane  may  be  ideal 
including  clean  airplane  surfaces,  a new 
engine,  a new  propeller,  an  unrestricted  air 
filter,  and  a company  test  pilot.  In  actual 
conditions  the  pilot  must  compare  these  book 
values  obtained  in  ideal  conditions  with  those 
in  which  he  finds  himself.  These  actual 
conditions  may  include  a dirty  airplane,  a 
slightly  used  engine,  a few  marks  on  the 
propeller,  a slightly  dirty  air  filter,  and  a 
less  than  perfect  pilot.  He  must  then 
evaluate  many  other  conditions  such  as  gross 
weight,  center  of  gravity,  wind,  temperature, 
humidity,  altitude,  etc.  for  comparison  with 
those  used  in  the  book  to  determine  his 
expected  flight  performance.  Finally,  he  must 
check  the  present  and  forecast  weather,  the 
terrain,  and  expected  traffic  density  and 
compare  them  with  an  estimate  of  his  own 
capability  before  determining  whether  or  not 
his  planned  flight  will  be  safe. 

Examples  such  as  these  requiring 
decisions  with  less  than  perfect  information 
are  available  in  all  areas  of  flight  activity. 
Furthermore,  every  decision  that  the  pilot 
makes  is  colored  by  physiological,  psycho- 
logical, and  social  pressures  that  are 
virtually  impossible  to  weigh  properly  on  the 
spot.  For  example,  just  as  persons  watching  a 
sporting  event  may  "see"  an  infraction  or  foul 
differently  depending  upon  their  vantage 
points  and  which  team  they  support,  a pilot 
may  be  influenced  to  view  the  weather  outlook 
or  his  own  abilities  differently  depending  on 
the  importance  or  value  he  assigns  to  a given 
flight.  The  person's  self-image  and  his  need 
to  maintain  his  external  image  largely 
determine  how  much  effect  different  values  or 
rewards  for  making  a flight  will  have  on  his 
judgment  of  his  ability  to  make  a safe  flight 
(Kogan  and  Wallach,  1964) * 


However,  if  it  were  merely  a matter  of 
teaching  flying  skills,  the  training  of  pilots 
to  operate  safely  in  our  complex  aviation 
system  would  be  a much  smaller  one  than  it  is. 
Unfortunately,  because  actual  conditions  are 
never  quite  the  same  as  those  used  to  develop 


Mental  weaknesses  in  some  pilots  may 
cause  them  to  be  susceptible  to  social 
pressures  that  result  in  les3  than  rational 
pilot  judgment.  Such  irrational  pilot  judg- 
ment is  characterized  by  3uch  unsafe  practices 
as  flying  under  bridges,  landing  on  busy 
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highways,  attempting  to  land  In  football 
stadiums,  and  flying  "formation"  on  other 
unsuspecting  pilots.  Potential  sources  of 
social  pressure  that  may  lead  to  these  types 
of  activities  include  peer  reactions,  fear  of 
failure,  censure  from  superiors  or  family 
members,  and  many  others  (Janis  and  Mann, 
1977). 

Although  it  may  be  a difficult  task 
(Fishbein  and  AJzen,  1975),  pilots  must  be 
tested,  not  only  for  their  knowledge,  skill, 
and  rational  judgment  capabilities,  but  also 
for  their  irrational  judgment  tendencies  a3 
these  apply  to  safe  flying.  It  is  apparent 
from  accident  statistics  that  new  approaches 
to  pilot  training  and  testing  are  needed  to 
improve  the  safety  and  effectiveness  of  civil 
and  military  pilots. 


be  a contributing  cause  or  factor."  Stati- 
stics from  the  National  Transportation  Safety 
Board  (NTSB)  Automated  Aircraft  Accident  and 
Incident  Information  System  from  1970  through 
1 974  were  used  in  this  analysis  (Jensen  and 
Benel,  1977).  Pilot  cause/factor3  from  the 
NTSB  data  were  classified  into  the  three 
behavioral  categories  given  above.  Then  the 
total  numbers  of  both  fatal  and  non-fatal 
accidents  during  the  five-year  period  were 
determined  for  each  of  these  behavioral 
categories.  The  results  of  these  analyses  are 
shown  in  Table  1. 

Table  1.  Number  and  percent  of  the  total 
general  aviation  accidents  in  which 
the  pilot  is  listed  as  a cause  or 
factor  between  1970  and  1974. 


Training  an 4 .Teaming  Effectiveness 


FATAL  NON-FATAL 


An  assessment  of  the  effectiveness  of 
current  pilot  training  and  testing  programs 
should  3tart  with  a categorical  analysis  of 
training  objectives  associated  with  the  end 
product:  a pilot  licensed  to  fly  under  a 
certain  set  of  regulations.  Civilian  training 
objectives  may  be  classified  under  three  sets 
of  behavioral  activities  as  follows: 

Procedural  Activities 

Communication  management 

Navigation  management 

Fuel  management 

Powerplant  management 

Vehicle  configuration  management 

Display  management 

Autopilot  management 

Perceptual-Motor  Activities 

Vehicle  control 

Distance,  speed,  altitude,  and  clear- 
ance judgments 

Hazard  detection  and  avoidance 

Communication 

Geographic  orientation 

Decisional  Activities 

Pilot  self-evaluation  of  skill,  know- 
ledge, physical,  and  psychological 
condition 

Navigation  planning 
Hazard  assessment 

Assessment  of  attention  requirements 
Assessment  of  aircraft  and  ground  sys- 
tem capabilities 
Mission  priority  adjustment 

A useful  next  step  in  the  examination  of 
the  effectiveness  of  current  pilot  training 
and  testing  programs  that  may  help  to  identify 
weaknesses  is  to  analyze  general  aviation 
accident  data  in  which  pilots  were  "found  to 


Procedural  264  (4.6$)  2230  (8.6$) 
Perceptual-Motor  2496  (43.8$)  14561  (56.3$) 
Decisional  2940  (51.6$)  9087  (35-1$) 


Although  these  sati3tics  of  pilot-caused 
accidents  reflect  the  influence  of  more 
factors  than  just  pilot  training  deficiencies 
alone,  examinations  of  these  data  provide 
valuable  indications  of  possible  weaknesses  in 
current  programs.  For  example,  a majority  of 
the  non-fatal  pilot-caused  accidents  (56.3 
percent)  were  the  result  of  faulty  perceptual- 
motor  behavior.  The  most  significant  factors 
here  (failure  to  maintain  flying  speed  and 
misjudgment  of  distance,  speed,  altitude,  or 
clearance)  represent  one  type  of  pilot 
judgment.  On  the  other  hand,  a majority  of 
the  fatal  pilot-caused  accidents  (51.6 
percent)  were  the  result  of  faulty  decisional 
behavior,  another  type  of  pilot  judgment.  The 
most  significant  factors  in  this  area  were  the 
familiar  "continued  VFR  into  known  adverse 
weather"  and  "inadequate  preflight  planning  or 
preparation." 

It  is  apparent  from  these  accident 
statistics  that  both  aspects  of  the  deciding 
function  are  important  to  safe  flight  and 
possibly  suffer  from  neglect  in  the  present 
training  and  testing  process.  However, 
because  it  suffers  from  greater  misunder- 
standing in  aviation  circles,  pilot  judgment 
as  represented  by  the  general  decisional 
activities  is  the  topic  of  concern  in  this 
paper.  Although  a significant  amount  of 
research  has  been  done  on  this  aspect  of 
judgment  in  recent  years  (Janis  and  Mann, 
1977)  no  one  has  specifically  examined  this 
judgment  problem  faced  by  the  pilot,  the 
flight  instructor,  and  the  pilot  examiner. 

There  appear  to  be  three  major  problems 
that  require  solution  before  major 
improvements  to  pilot  training  and  evaluation 
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can  be  realized  in  this  area.  The  first  is 
the  establishment  of  a common  definition  of 
Judgment  as  it  applies  to  flying.  At  present, 
even  though  the  term  is  used  repeatedly  in 
aviation  circles  and  FAA  examiners  are 
required  to  evaluate  candidates  on  the  basis 
of  judgment,  no  such  definition  exists. 

The  second  major  problem  is  to  determine 
whether  or  not  pilot  judgment  can  be  taught, 
and  if  so,  how  can  one  best  teach  it.  Because 
3ome  aspects  of  pilot  judgment  are  closely 
akin  to  personality  characteristics,  they  may 
be  difficult  to  modify.  It  may  be  necessary 
to  use  testing  and  selection  procedures  to 
improve  aviation  safety  and  effectiveness  from 
these  standpoints.  Other  aspects  of  pilot 
judgment  are  more  easily  modified  through 
systematic  training  procedures. 

The  third  major  problem  is  to  determine 
whether  or  not  pilot  judgment  can  be  evaluated 
reliably,  meaningfully,  and  objectively. 
Because  judgment  is  primarily  a mental 
process,  it  may  be  difficult  to  evaluate  in 
any  reliable  way.  On  the  other  hand, 
behavioral  events  frequently  have  been  used  to 
infer  mental  activity.  Although  personality 
tests  have  proved  to  be  somewhat  unreliable, 
research  results  using  these  instruments  may 
be  useful  in  the  development  of  instruments 
for  evaluating  and  predicting  judgmental 
behavior  (Fishbein  and  Ajzen,  1975). 

Judgment  Definition 

As  indicated  above,  the  word  judgment  has 
been  used  to  describe  two  somewhat  different 
mental  processes  in  aviation.  Perhaps  its 
ao3t  common  usage  has  been  to  describe  the 
mental  activity  that  takes  place  at  the 
perceptual-motor  level.  The  second  describes 
the  mental  activity  involved  in  choosing  a 
course  of  action  from  among  several 
alternatives.  Obviously,  this  second  usage  of 
the  term  is  similar  to  the  first  in  that  both 
involve  making  choices. 

However,  there  is  a basic  difference. 
The  first  refers  to  highly  learned  perceptual 
responses  that  must  be  made  in  a very  short 
time,  in  some  cases  continuously.  The  second, 
refers  to  cognitive  decisions  for  which  set 
procedures  have  not  been  established  or  may 
have  been  forgotten.  Flight  instructors  have 
used  various  terms  referring  to  this  type  of 
judgment  including  "headwork,"  "thinking 
ahead,"  and  "staying  ahead  of  the  aircraft." 
Usually,  more  time  is  available  to  evaluate 
the  situation,  a larger  number  of  possible 
courses  of  action  must  be  considered,  and 
there  is  a greater  degree  of  uncertainty 
concerning  the  existing  situation  and  possible 
outcomes  than  is  the  case  in  perceptual 
judgments.  For  these  reasons,  cognitive 
judgments  have  been  the  source  of  greater 


misunderstanding  in  pilot  training  and 
evaluation. 

These  two  aspects  of  Judgment  may  be 
considered  as  two  ends  of  a continuum  based  on 
cognitive  complexity  and  decision  time.  One 
such  representation  is  shown  in  Figure  1.  At 
one  end  of  the  continuum,  are  the  common 
perceptual  judgments  of  distance,  altitude, 
speed,  and  clearance.  These  perceptual 
Judgments  are  less  complex  in  that  they 
involve  fewer  pieces  (frequently  one)  of 
fairly  accurate  information  from  which 
responses  are  determined  with  highly  learned 
motor  behavior.  They  may  require  simple 
responses  but  frequently  call  for  immediate 
control  movement. 

At  the  other  end  are  what  might  be  called 
cognitive  judgments.  As  described  above, 
these  judgments  are  very  complex  in  that  they 
usually  involve  a large  number  of  relevant 
pieces  of  highly  probabilistic  information, 
they  usually  require  the  specification  of  and 
choice  from  among  several  alternatives,  and 
they  are  frequently  affected  by  emotions, 
values,  and  social  pressures.  In  addition, 
cognitive  judgments  usually  permit  some 
deliberation  before  a control  response  is 
required.  The  remainder  of  this  paper  is 
concerned  with  this  aspect  of  pilot  judgment. 

Perceptual  Cognitive 

Judgments  Judgments 


Increasing  Cognitive 

Complexity 

Figure  1.  Judgment  Continuum  Based  on  Cogni- 
tive Complexity  and  Decision  Time 

Cognitive  Judgment  Definition.  Consider- 
ing these  factors  a candidate  definition  of 
cognitive  judgment  in  flying  airplanes  is: 

The  ability  to  search  for  and  establish 
the  relevance  of  all  available  information 
regarding  a situation,  to  specify 
alternative  courses  of  action,  and  to 
determine  expected  outcomes  from  each 
alternative. 

The  motivation  to  choose  and  authorita- 
tively execute  a suitable  course  of  action 
within  the  time  frame  permitted  by  the 
situation. 

Where: 

1.  "Suitable"  is  an  alternative  consis- 
tent with  societal  norms. 

2.  "Action"  includes  no  action,  some  ac- 
tion, or  action  to  seek  more  infor- 
mation. 
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The  first  part  of  the  definition  refers 
to  intellectual  abilities.  It  depends  upon 
human  capabilities  to  sense,  store,  retrieve, 
and  integrate  information.  This  function  is 
what  Van  Dam  (in  Jensen  and  Benel , 1977)  calls 
the  "discriminating  ability"  in  professional 
pilots.  In  signal  detection  theory  it  is 
called  detectability  ( d * ) . It  is  purely 
rational  and  could  be  stated  mathematically. 
If  it  were  possible  to  separate  this  part  of 
human  judgment  from  the  second  part  (which  it 
is  not),  the  mind  would  solve  problems  in  much 
the  same  way  as  a computer.  This  is  not  to 
say  that  it  would  be  error  free.  It  uses 
probabilistic  information  and  is  dependent 
upon  the  amount,  type,  and  accuracy  of 
information  stored  a3  well  as  inherent  and 
learned  capabilities  to  process  information. 

The  second  part  of  the  definition  refers 
to  motivational  tendencies.  The  emphasis  is 
on  the  directional  aspects  of  motivation 
rather  than  the  aspects  of  motivation  dealing 
with  intensity.  It  says  that  a part  of  human 
judgment  is  based  upon  bias  factors  (costs  and 
payoffs)  or  tendencies  to  use  less  than 
rational  information  (defined  by  society)  in 
choosing  courses  of  action.  Society  would 
probably  consider  the  use  of  any  information 
other  than  that  required  to  define  the  safety 
risk  (e.g.,  monetary  gain,  gain  in  self- 
esteem, adventure  seeking,  etc.  ) as  less 
than  rational.  This  part  of  human  judgment  is 
called  the  response  bias  (B)  in  signal 
detection  theory.  It  is  what  Van  Dam  (in 
Jensen  and  Benel,  1977)  has  called  the 
"response  pattern"  of  the  professional  pilot. 
If  properly  developed,  this  part  of  human 
judgment  would  tend  to  halt  the  use  of 
information  not  directly  related  to  the  safety 
of  the  flight  and  to  direct  the  pilot's 
decision  toward  the  use  of  rational  processes. 

JUDGMENT  TRAINING 
Can  Pilot  Judgment  Be  Taught? 

The  first  question  to  be  addressed 
following  the  establishment  of  the  definition 
is  whether  or  not  pilot  judgment,  as  defined, 
can  be  modified  through  training.  The  paucity 
of  judgment  training  guidelines  in  pilot 
training  and  training  research  literature 
leads  one  to  doubt  that  judgment  can  be 
taught.  Literature  and  syllabi  commonly  used 
in  flight  instructor  courses  contain  large 
sections  on  how  to  teach  the  motor  skills  of 
flying  but  very  little  on  how  to  teach  pilot 
judgment  (3ee  the  FAA’s  Aviation  Instructor 1 s 
Handbook.  1977).  The  typical  private  pilot 
course  offers  a scattering  of  judgmental 
instruction  in  the  areas  of  weather  avoidance 
and  power-plant  emergencies  but  no  systematic 
judgmental  training. 

However,  there  is  evidence  in  aviation 
showing  that  at  least  one  form  of  judgmental 


training,  assigning  procedures  for  every 
conceivable  situation  that  might  arise,  may  be 
effective.  In  the  military  these  are  referred 
to  a3  "Boldface"  training  procedures.  Demon- 
strations by  American  Airlines  (Gibson,  1969) 
and  by  Trans  World  Airlines  (Trans  World 
Airlines,  1969)  offer  convincing  support  for 
the  conclusion  that  complex  simulators  are 
effective  both  for  the  training  and  testing  of 
pilots  using  these  procedures. 

Looking  outside  the  field  of  aviation  one 
finds  other  evidence  indicating  that  judgment 
may  be  taught.  For  example,  although  the 
theory  of  signal  detection  (TSD)  was  not 
designed  specifically  to  handle  cognitive 
judgments,  many  of  its  methods  can  be  U3ed  to 
explain  and  perhaps  even  modify  pilot  judgment 
behavior.  TSD  divides  an  individual's 
decision  behavior  into  two  components 
representing  his  sensitivity  (d' ) and  his 
response  criterion  or  bias  (B),  roughly 
corresponding  to  the  two  aspects  of  our 
judgment  definition. 

The  sensitivity  is  affected  both  by  the 
physical  value  of  the  stimuli  in  the  situation 
(signal  vs  background  noise)  and  the  quality 
of  the  sensory  apparatus  of  the  observer.  In 
cognitive  judgment  this  is  the  intellective 
component.  On  the  other  hand,  the  response 
criterion  represents  the  point  in  the 
signal-to-noise  distribution  at  which  the 
observer  is  willing  to  say  "signal."  It  is 
the  amount  of  information,  in  the  presence  of 
noise,  needed  to  tip  the  decision  one  way  or 
the  other.  It  is  influenced  by  motivation, 
knowledge  of  the  signal’s  probability  of 
occurrence,  and  the  costs  and  payoffs 
attendant  with  a given  response.  In  cognitive 
judgment  the  response  criterion  is  the 
motivative  component. 

The  response  criterion  can  be  manipulated 
through  a wide  range  of  values  by  adjusting 
probabilities,  costs,  and  payoffs  (Birdsall, 
1955).  We  can  infer  from  the  vast  amount  of 
psychophysical  decision  data  that  cognitive 
judgments  can  be  modified  in  a similar  way. 
Decision  biases,  attitudes,  risk  tendencies, 
consideration  for  passenger  safety,  and  pilot 
motivation  can  and  are  being  taught  by  the 
flight  instructor  by  example,  if  not  by 
design,  at  all  levels  of  pilot  training. 
These  tendencies  are  taught,  perhaps 
unconsciously,  by  the  assignment  of  proba- 
bilities, costs,  and  payoffs  to  actions  of  the 
student  by  the  instructor. 

Although  TSD  says  that  the  sensitivity 
component  is  quite  stable  for  a given 
individual,  there  is  a growing  field  of 
research  indicating  that,  if  considered  as  the 
intellective  component  of  cognitive  judgment, 
sensitivity  can  be  modified  as  well.  For 
example,  attempts  have  been  made  to  discover 
the  mental  processes  that  are  used  by  expert 
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Judges  such  as  stock  brokers,  livestock 
judges,  and  medical  diagnosticians  in  making 
their  decisions  (Shanteau  and  Phelps,  1977; 
Slovic,  1969',  Anderson,  1969;  Hoffman,  Slovic, 
and  Rorer,  1968).  The  hypothesis  is  that  if 
models  of  the  mental  processes  used  by  these 
experts  in  decision-making  were  available, 
they  could  be  used  in  training  others  to  use 
similar  processes.  In  each  of  the  areas 
studied,  judgmental  training  traditionally 
occurs  over  a fairly  long  apprenticeship 
program  in  which  the  trainee  observes  the 
expert  make  decisions  and  learns  by  this 
observation.  However,  as  in  aviation,  because 
of  the  complexity  of  the  information  used  to 
make  decisions,  observation  or  even  trial  and 
error  are  inefficient  training  methods. 

The  research  on  the  motivative  aspect  of 
cognitive  judgment  also  indicates  that 
training  can  have  a beneficial  effect.  The 
major  research  efforts  in  this  area  are 
reported  by  Janis  and  Mann  (1977).  These 
authors,  speaking  from  a clinical  perspective, 
begin  with  the  assumption  that  psychological 
stress  is  a frequent  cause  of  errors  in 
decision  making.  They  say  that  stress  arises 
from  at  least  two  sources.  First,  the 
decision-maker  is  concerned  about  the  material 
and  social  losses  he  may  suffer  from  whichever 
course  of  action  he  chooses,  including  the 
costs  of  failing  to  live  up  to  prior 
commitments.  Second,  he  recognizes  that  his 
reputation  and  self-esteem  as  a competent 
uecision-maker  are  at  stake.  The  more  severe 
the  anticipated  losses,  the  greater  the 
stress. 

Janis  and  Mann  have  constructed  a 
"conflict-theory"  model  of  decision-making 
postulating  that  the  way  we  resolve  a 
difficult  choice  is  determined  by  the  presence 
or  absence  of  three  conditions:  "awareness  of 
risks  involved,"  "hope  of  finding  a better 
solution,"  and  "time  available  in  which  to 
make  the  decision."  They  have  developed 
several  clinical  procedures  to  improve 
decision-making  under  the  titles,  "awareness- 
of-rationalizations,"  "emotional  role  play- 
ing," "balance  sheet,"  and  "outcome 
psychodrama."  They  report  that  these 
procedures  have  demonstrated  effectiveness  in 
changing  decision-making  tendencies  and  in 
attitude  modification. 


The  need  for  pilot  judgmental  training 
has  been  established  for  all  levels  of  flight 
instruction.  Without  a systematic  judgmental 
training  program,  good  pilot  judgment  is 
acquired  by  the  cautious  and  the  lucky  over 
years  of  flying  experience  in  many  varied 
situations.  Our  task  as  aviation  educators 
using  systematic  judgmental  training  tech- 
niques should  be  to  compress  a lifetime  of 
flying  experience  into  a relatively  short 


training  program  to  instill  good  pilot 
judgment  into  the  emerging  private  or  military 
pilot. 

The  evidence  presented  in  the  preceding 
section  indicates  that  many  aspects  of  pilot 
judgment  can  be  taught.  The  questions  that 
remain  are:  what  approaches  should  be  taken  to 
implement  pilot  Judgmental  training  and  what 
techniques  should  be  used  to  evaluate  the 
level  of  judgment  possessed  by  a pilot  or 
flight  student.  This  section  presents  a 
systematic  approach  to  pilot  training  empha- 
sizing judgmental  instruction. 

Before  proceeding,  it  is  necessary  to 
establish  some  definitions  and  constraints 
involved  in  this  approach.  First,  training 
and  education,  which  have  been  distinguished 
elsewhere  (Glaser,  1962),  will  be  considered 
equivalent  and  defined  as  the  "systematic 
acquisition  of  skills,  rules,  concepts,  or 
attitudes  that  results  in  improved  performance 
in  another  environment"  (Goldstein,  1974). 
The  "systems  approach,"  a term  that  has  been 
used  and  abused  in  many  ways,  should,  in  this 
training  context,  emphasize  the  specification 
of  instructional  objectives,  precisely 
controlled  learning  experiences  to  achieve 
these  objectives,  criteria  for  performance, 
feedback  within  the  system,  and  a recognition 
of  the  interaction  among  system  components. 

In  addition  to  these,  the  approach  to 
pilot  judgmental  training  should  consider  the 
following  constraints:  the  cost  and  time 
required  of  both  student  and  instructor,  the 
qualifications  required  of  the  flight 
instructor  and  examiner,  and  the  safety 
requirements  to  administer  such  a program. 
Finally,  although  a systems  approach  is  used 
to  develop  the  training  context,  the  major 
burden  of  judgmental  training  falls  directly 
on  the  flight  instructor.  He  or  she  is 
responsible  for  the  creation  and  use  of 
innovative  situational  teaching  techniques. 

An  Instructional  Model.  A model  of  an 
instructional  system  adapted  from  one 
developed  by  Goldstein  (1974)  is  useful  for 
the  establishment  of  the  system  context  for 
pilot  judgmental  training.  This  model  shown 
in  Figure  2,  presents  five  basic  interrelated 
phases  in  a closed-loop  instructional  system: 
assessment,  selection,  development,  training, 
and  evaluation.  All  five  phases  are  needed  to 
accomplish  the  goals  of  a systematic  approach 
to  any  instructional  program.  The  feedback 
from  the  evaluation  phase  to  the  assessment 
phase  indicates  that  an  instructional  system 
is  never  complete.  It  needs  continual 
adjustment  based  on  the  results  of  the 
evaluation  phase  and  inputs  from  the 
environment. 

The  assessment  phase  consists  of  the 
establishment  of  the  instructional  need  and  a 
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Figure  2.  A Model  for  Pilot  Judgmental  Training 


derivation  of  behavioral  objectives.  An 
assessment  of  the  need  for  pilot  Judgmental 
instruction  first  requires  an  anaylsis  of  the 
present  and  future  requirements  for  pilot 
judgment  in  the  flying  environment.  Second, 
this  assessment  requires  an  analysis  of  the 
judgmental  task  from  the  behavioral  stand- 
point. Third,  it  requires  an  analysis  of  human 
attributes  necessary  to  perform  the  judgmental 
tasks.  These  three  analyses  provide  the  basis 
for  the  development  of  judgment  behavioral 
objectives. 

Behavioral  objectives  should  specify  what 
the  trainee  will  be  able  to  accomplish  when  he 
successfully  completes  the  instructional 
program.  They  should  also  indicate  the  condi- 
tions under  which  the  performance  must  be 
maintained  and  the  standards  by  which  the 
trainee  will  be  evaluated.  Thus,  they  provide 
direct  inputs  both  into  the  evaluation  phase 
and  the  development  phase  of  the  instructional 
model. 

The  selection  phase  consists  of  a program 
of  psychological  tests  primarily  aimed  at  the 
identification  of  persons  likely  to  exhibit 
irrational  judgment  behavior  during  their 
flying  career.  For  obvious  reasons,  this 
phase  would  be  easier  to  apply  in  military 
settings  than  civilian  settings,  even  though 
accident  statistics  have  shown  that  such 
selection  methods  might  save  lives.  The 


necessary  instruments  for  effective  appli- 
cation of  this  phase  are  yet  to  be  developed. 

The  development  phase  consists  of  the 
establishment  of  a training  program  to  achieve 
the  behavioral  objectives.  The  development  of 
this  program  requires  a blend  of  learning 
principles  and  media  selection  based  on  the 
skills,  concepts,  and  attitudes  that  are  to  be 
transferred  to  the  operational  flying 
environment.  The  learning  principles  are 
integrated  and  matched  with  appropriate 
training  media  in  the  Training  Effectiveness 
Cost  Effectiveness  Prediction  (TECEP)  model 
(Braby,  Michelli,  Morris,  Okraski , 1972). 

The  training  phase  consists  of  both 
instructor  and  student  training  programs. 
Because  judgmental  instruction  requires  the 
use  of  situational  teaching  techniques, 
instructors  will  need  special  training  to 
administer  these  techniques.  The  instructor 
training  program  will  also  serve  as  the  source 
of  much  of  the  situational  material  used  in 
judgmental  training. 

The  evaluation  phase  consists  of  the 
systematic  measurement  of  changes  brought 
about  by  the  training  program.  Thus,  the 
evaluation  phase  requires  the  establishment  of 
measures  of  success  (criteria),  based  on  the 
behavioral  objectives,  and  the  measurement  of 
judgmental  behavior  both  before  and  after  the 
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training  process.  Although  rarely  seen,  data 
from  these  evaluations  are  vital  to  the 
success  of  any  training  program.  As  Goldstein 
(p.  23)  points  out,  "...instructional  pro- 

grams are  research  efforts  that  must  be 
massaged  and  treated  until  the  required 
results  are  produced." 

In  pilot  Judgmental  evaluation  there  also 
is  a requirement  for  the  assessment  of 
judgmental  capabilities  and  tendencies 
relative  to  an  absolute  standard  determined  by 
what  society  expects  of  pilots.  This 
evaluation  would  be  included  as  a part  of  the 
pilot  certification  process.  Such  an 
evaluation  requires  a knowledge  of  societal 
demands  concerning  pilot  judgmental  capabi- 
lities, at  least  qualitative  criteria  against 
which  to  judge  the  candidate,  and  an  unbiased 
observation  of  performance  to  determine 
whether  or  not  the  candidate  meets  the 
criteria. 

Some  Learning  Principles 

Because  of  the  common  misapplication  of 
some  well  established  learning  principles  in 
many  training  programs,  a discussion  of  these 
principles  as  applied  to  pilot  Judgmental 
training  is  needed.  Perhaps  the  most  popular 
of  these  is  the  assumption  that  the  best  way 
to  learn  an  activity  is  to  practice  that 
activity  (Gagne,  1962).  Thi3  assumption  is 
rooted  in  much  of  the  educational  literature 
and  is  often  identified  by  the  catch-phrase 
"learning  by  doing."  Gagne  points  out  that  it 
may  also  be  a generalization  of  the  research 
on  the  conditioned  response  in  which  learning, 
particularly  in  animals,  appears  to  have 
» occurred  only  after  a response  (practice)  has 

been  made. 

However,  Gagne  argues  that  practice  is 
not  an  effective  training  method  by  itself, 
even  for  the  acquisition  of  such  motor  skills 
as  field  gunnery.  He  says  that  "instruction 
about  the  correct  sighting  picture  for  ranging 
is  more  effective  in  bringing  about  improved 
performance"  than  is  practice  on  the  task. 
The  point  is  that  training  should  emphasize 
the  principles  and  procedures  (thought 
processes)  involved,  and  practice  should  be 
directed  to  take  advantage  of  these  principles 
or  take  a minor  role.  If  this  is  the  proper 
emphasis  for  teaching  motor  skills,  it  is  even 
more  important  in  the  teaching  of  judgmental 
skills  which  are  more  highly  rooted  in  thought 
processes. 

A second  learning  principle  that  is 
frequently  misapplied  in  training  situations 
is  variously  called  reinforcement,  feedback, 
or  knowledge  of  results  during  practice.  This 
principle  has  been  found  to  be  most  effective 
in  choice  behavior.  However,  Gagne  points  out 
that  some  manipulations  that  artificially 
improve  feedback  during  practice  failed  to 


show  reliably  better  transfer  to  the 
operational  environment  and  others  showed 
negative  transfer.  Apparently  the  form  of  the 
feedback  Is  Important. 

Any  beginning  flight  student  will  tell 
you  that  the  usual  feedback  Information  such 
as  "you  did  it  right"  or  "you  did  It  wrong"  Is 
almost  useless.  The  time  period  between 
trials  and  feedback  may  be  long,  It  Is  often 
cluttered  with  interferrlng  information,  and 
the  trials  themselves  are  often  so  complex 
that  the  student  may  learn  very  little  from 
such  a response  by  his  flight  instructor.  The 
student  really  needs  to  know  why  he  did  it 
right  or  wrong.  He  needs  to  know  what  rules 
he  should  have  followed  and  where  he  strayed 
from  those  rules.  Although  practice  and 
right/wrong  types  of  feedback  may  be  useful  In 
some  training  situations,  they  3hould  be 
de-emphasized  in  favor  of  these  "thought" 
oriented  teaching  principles  in  all  types  of 
pilot  training,  but  especially  in  judgmental 
training. 

Judgmental  Training  Media 

Because  of  the  nature  of  the  subject 
matter  to  be  taught  (i.e.,  attitudes,  prin- 
ciples, and  motivations),  the  primary  load  of 
pilot  judgmental  training  must  be  borne  by  the 
flight  instructor.  Practice  and  conventional 
self-teaching  techniques  (e.g.,  solo  flying  in 
a practice  area)  are  highly  inefficient 
methods  for  imparting  these  concepts.  The 
following  is  a discussion  of  some  suggested 
judgmental  training  media  and  techniques  that 
could  be  applicable  to  pilot  training  in  civil 
and  military  aviation. 

Ground  School ■ There  are  a number  of 
excellent  ways  that  pilot  judgment  could  be 
taught  from  the  perspective  of  the 
conventional  ground  school.  To  afford  it 
proper  emphasis,  it  is  suggested  that  judgment 
should  be  given  a special  section  of  ground 
school  with  the  same  status  a,  meteorology, 
navigation,  and  Federal  Air  Regulations.  This 
section  could  include  lectures  and/or  discus- 
sions of  aviation  accident  senarios  in  which 
the  pilot  was  a cause  or  factor,  interactive 
movies,  video  tapes,  slide  presentations 
requiring  student  judgmental  responses  at 
critical  points  in  flight  senarios,  and 
independent  study  of  the  principles  involved 
in  good  pilot  judgment. 

In  addition,  this  ground  school  section 
could  include  instruction  in  information 
integration  and  subjective  probability  estima- 
tion (Goldberg,  1968).  Judgmental  behavior  in 
expert  judges  is  characterized  by  chunking,  or 
the  formation  of  clusters  of  stimulus 
attributes  and  response  alternatives,  for 
economy  in  the  thought  process.  Ground  school 
students  could  be  taught  to  use  these 
processes  in  their  judgmental  activity.  The 
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instructor  would  show  how  various  types  of 
probabilistic  information  such  as  weather 
forcast3,  predicted  aircraft  system  malfunc- 
tions, and  predicted  Air  Traffic  Control 
problems  should  be  combined  in  making  flying 
decisions. 

The  instructor  would  teach  the  student 
how  to  "think  ahead"  or  anticipate  decisions 
that  might  have  to  be  made  later  resulting 
from  present  choices  of  action.  Such 
anticipation  permits  the  gathering  of  relevant 
information  under  lower  levels  of  stress,  when 
errors  are  less  frequent  than  later  in  the 
flight  when  time-to-decide  may  become  a error 
causing  factor.  This  section  of  ground  school 
could  also  include  decision-making  training 
using  procedures  suggested  by  Janis  and  Mann 
(1977)  such  as  ’’balance  sheet"  and  "emotional 
role  playing." 

Computer-Assisted  Instruction.  An  in- 
structional technique  that  holds  unusual 
promise  for  pilot  judgmental  training  and 
evaluation  is  computer-assisted  instruction 
(CAI).  The  great  advantage  of  the3e  systems 
is  that  they  can  teach  principles  and  then 
permit  the  student  to  participate  in  decision 
processes,  a highly  effective  learning 
technique  (Fishbein  and  Ajzen,  1975).  The 
disadvantage  of  these  systems  in  the  past  has 
been  their  limited  availability  and  high  cost. 
However,  recent  advances  in  technology  are 
making  them  available  at  a relatively  low  cost 
(Trollip  and  Ortony,  1 977 ) - 

Although  CAI  programs  are  available  in 
several  forms,  the  dialogue  systems  that 
permit  student-computer  interaction  unre- 
stricted by  preset  response  alternatives 
(Alpert  and  Bitzer,  1970),  show  the  greatest 
potential  for  application  to  judgmental 
instruction.  These  systems  depend  upon  a set 
of  stored  algorithms  that  are  used  by  the 
computer  to  construct  a great  variety  of 
responses  to  student  questions.  In  addition, 
student  responses  are  not  limited  to  exact 
duplicates  of  prestored  expected  responses. 
The  program  recognizes  a variety  of  student 
responses  and  is  able  to  proceed  accordingly. 

Although  practice  and  feedback  are 
frequently  used  concepts  in  CAI  programs, 
these  could  be  augmented  by  presenting 
principles  and  reasons  for  taking  certain 
courses  of  action.  In  judgmental  training  the 
student  could  be  presented  with  a flight 
situation  requiring  Judgment.  He  could  then 
be  a3ked  to  respond  by  listing  all  of  his 
alternatives  and  the  factors  affecting  each. 
He  could  even  be  asked  to  estimate  the 
probability  of  success  for  each  alternative. 

The  computer  could  then  examine  the 
flight  experience  data  on  the  student  (entered 
previously)  and  the  stored  accident  statistics 
from  similar  circumstances.  The  computer 


would  then  respond  with  comments  on  the 
appropriateness  of  the  student's  responses, 
the  alternatives  that  may  have  been  omitted, 
and  the  principles  that  should  have  been 
followed  in  making  the  decision.  The  program 
could  then  branch  to  another  problem,  the 
difficulty  of  which  would  be  based  on  the 
level  of  judgmental  capability  evidenced  by 
the  student's  responses  to  the  previous 
problem. 

Complexity,  realism,  and  time  constraints 
could  be  included  in  the  judgmental  task  by 
the  addition  of  a simple  flight  hand- 
controller  and  an  airplane  symbol  with  a map 
on  the  screen.  The  purpose  of  this  controller 
would  be  to  provide  indications  of  progress 
toward  a destination  and  time  available  for 
the  decision,  not  for  instruction  in  flight 
control . 

CAI  has  many  advantages  not  commonly 
associated  with  other  instructional  systems. 
The  most  important  of  these  is  individuali- 
zation of  instruction.  It  can  adapt  to  the 
specific  needs  of  the  individual  and  interact 
at  his  current  level  of  ability  (Goldstein, 
197*0.  Second,  the  unencumbered  reinforcement 
capabilities  of  CAI  are  a i eal  benefit  to  the 
student.  It  has  no  personality  or  ulterior 
motives  to  clash  with  those  of  the  student. 
Third,  CAI  systems  do  not  require  the  presence 
of  a teacher,  although  it  may  be  beneficial  to 
have  one  present  for  occasional  consultation. 
Fourth,  they  permit  standardization  of  in- 
struction across  a wide  area.  One  central 
computer  could  potentially  support  terminals 
at  every  pilot  instructional  center  in  the 
United  States  at  a relatively  low  cost.  No 
student  would  be  handicapped  by  a bad 
instructor  who  underscores  weaknesses  in  the 
simulation.  Fifth,  data  gathered  from  student 
responses  could  be  stored  for  as  long  as 
necessary  for  use  in  updating  instructional 
programs  or  in  evaluating  individual  pilot 
judgmental  capabilities. 

CAI  also  has  a number  of  limitations  that 
may  impact  judgmental  training.  First,  very 
little  is  known  about  the  effectiveness  of  the 
instructional  techniques  described  above. 
Research  is  still  needed  to  determine  how  to 
program  such  an  instructional  system  most 
effectively.  Second,  large  outlays  of  money 
would  be  required  for  hardware  to  implement 
such  a program.  Third,  some  users  might 
object  to  the  requirement  for  communication 
with  the  computer  via  a keyboard,  although 
keyboards  are  rapidly  becoming  a part  of  the 
pilot's  way  of  life.  Fourth,  Goldstein  (1974) 
expresses  concern  for  the  effects  of  a 
machine-oriented  learning  environment  on 
satisfaction,  motivation,  and  development. 

Flight  Simulation,  Of  the  various 
alternatives  available  for  pilot  tr  lining,  it 
is  apparent  that,  for  many  operators,  flight 


simulators  may  be  the  most  viable  for  all 
types  of  pilot  training.  The  education  of 
pilots  for  military  and  airline  applications 
has  become  increasingly  dependent  upon 
ground-based  aircraft  simulators. 

In  some  ways  judgmental  training  in  a 
simulator  environment  would  be  more  cumbersome 
than  in  ground  school  or  CAI  because,  at  least 
in  current  practice,  it  depends  upon  the 
instructor  to  create  the  simulated  flight 
situation  primarily  through  verbal  communi- 
cation. Nevertheless,  the  simulated  flight 
environment  provides  an  additional  opportunity 
to  teach  judgmental  principles,  if  properly 
structured,  in  a somewhat  more  realistic 
environment  than  ground  school  or  CAI  can 
provide. 

Probably  the  best  way  to  begin  judgmental 
training  in  the  simulator  is  to  use  the 
airline  approach  i.e.,  teaching  procedures 
that  are  to  be  followed  in  each  situation  that 
departs  from  normal  flight.  This  includes 
system  failure  detection  as  well  as  establish- 
ing courses  of  action  to  correct  or  counter 
system  failures.  Principles  involved  as  well 
as  corrective  procedures  would  be  taught 
according  to  this  method,  and  appropriate 
judgmental  performance  measures  could  be 
developed. 

The  simulator  instruction  could  also 
include  the  creation,  by  the  flight 
instructor,  of  judgment-demanding  situations 
that  do  not  involve  the  failure  of  systems. 
These  situations  would  demand  decisions  such 
as  whether  or  not  to  continue  a flight  into 
deteriorating  weather,  decisions  about 
passenger  demands  for  landing  at  an  unfamiliar 
alternate  airport,  decisions  about  weight  and 
balance  considering  field  conditions,  density 
altitude,  etc.  In  all  cases  the  instructor 
would  ask  the  student  to  state  several 
alternatives  available  to  him  and  also  to 
state  which  he  would  choose.  These  situations 
could  be  developed  from  NTSB  accident  briefs, 
and  they  could  be  a part  of  the  flight 
instructor's  simulator  judgmental  instruction 
package. 

Simulator  judgmental  instruction  should 
be  treated  as  a serious  and  vital  part  of  the 
flight  student's  training.  The  simulator  must 
be  treated  as  an  important  training  aid  just 
a3  the  airplane  and  the  blackboard  are 
treated.  The  instructor  ha3  the  opportunity 
and  responsibility  to  instill  serious,  mature 
flight  attitudes  in  hi  3 students  by  his 
approach  to  judgmental  training.  The 
simulator  provides  an  outstanding  medium  for 
teaching  a 3tudent  good  judgment.  But  the 
training  will  only  be  as  valuable  as  the 
instructor's  approach  to  simulator  instruction 
is  serious. 


The  Airplane.  Of  all  the  media  avail- 
able, the  airplane  is  probably  the  most 
difficult  to  use  for  direct,  systematic 
judgmental  training.  The  reason  is  that  for 
the  sake  of  safety,  convenience,  and  cost  most 
judgmental  problems  must  be  halted  before  the 
student  sees  the  final  consequences  of  his 
decisions.  He  frequently  must  take  the 
instructor  at  his  word  that  his  decision  would 
have  resulted  in  a safe  or  unsafe  situation. 
However,  the  airplane  offers  special 
opportunities  for  judgmental  instruction 
because  the  environment  is  more  realistic,  it 
is  more  meaningful,  and  therefore,  it  is  more 
likely  to  cause  a more  permanent  behavioral 
change  in  the  student  than  other  training 
media. 

Everything  that  has  been  said  about 
instructor  attitudes  and  approaches  to 
judgmental  training  applies  doubly  when 
actually  flying  the  airplane.  Effective 
judgmental  instruction  in  the  airplane 
requires  a consistent,  disciplined  flight 
instructor  who  always  follows  the  rules  that 
the  student  is  expected  to  follow,  or  provides 
a good  explanation  for  why  he  deviates  from 
them.  It  also  requires  that  the  instructor 
follow  the  learning  principles  stated  earlier 
i.e.,  that  practice  and  feedback  are 
beneficial  only  when  accompanied  by  direction 
and  explanations. 

Judgmental  instructions  in  the  airplane 
should  take  the  form  of  simulated  situations 
created  by  the  instructor  requiring  the  use  of 
judgment.  Such  activities  should  be  inter- 
spersed throughout  the  flight  training 
program.  Such  instruction  is  already  being 
done  to  some  extent  through  training  in 
simulated  engine  failures,  other  system 
failures,  and  all  types  of  stalls.  This 
training  could  be  expanded  to  include  many  of 
the  hypothetical  situations  discussed  above. 
Portions  of  such  simulated  situations  could  be 
a part  of  every  instructional  flight. 

It  is  the  flight  instructor's  responsi- 
bility to  teach  the  student  that  it  is  not 
socially  demeaning  to  refuse  to  fly  or  to  turn 
around  in  the  face  of  deteriorating 
circumstances.  Such  situations  should  be  made 
to  occur  several  times  during  the  student's 
instruction  program  in  the  airplane.  Pilots 
have  often  said  that  it  is  most  difficult  to 
turn  around  the  first  time.  In  this  regard, 
it  is  important  to  teach  the  student  how  to 
avoid  the  tremendous  social  pressure  that  a 
group  of  important  passengers  can  exert.  The 
pilot  must  be  taught  to  isolate  himself  from 
his  flight  naive  passengers  in  all  important 
deci sions. 

Finally,  often  one  of  the  most  difficult 
evaluations  a pilot  has  to  make  is  the 
self-evaluation  of  his  own  skill,  knowledge, 
and  Judgmental  capability  relative  to  a 


proposed  night.  To  assist  himself  In  this 
regard  he  should  develop  a list  of  personal 
limitations  on  flight  procedures  based  on  his 
own  capabilities.  These  limitations  must  be 
applicable  to  all  flights  regardless  of  who 
the  passengers  are  or  how  much  they  are 
willing  to  pay  him  to  make  the  flight.  They 
should  be  invoked  during  a rational  moment, 
and  the  pilot's  resolution  should  be  strong 
enough  to  withstand  the  enormous  social 
pressure  to  deviate  from  them  either  before  or 
during  a given  flight. 

Situational  Emergency  Training 

The  Air  Force  has  begun  a research 
program  (Thorpe,  Martin,  Edwards,  and  Eddows, 
1976)  aimed  at  improving  pilot  decisional 
processes  during  emergency  situations. 
Although  the  goals  of  this  program  are  more 
limited  than  those  of  judgmental  training 
described  above  for  civil  aviation,  the 
approaches  suggested  are  very  similar.  The 
proposed  training  program  being  studied, 
called  "Situational  Emergency  Training"  (SET), 
is  designed  for  the  F-15  to  replace  the 
traditional  "Boldface"  procedures  of  other 
USAF  weapons  systems.  Although  Boldface 
procedures  are  effective  in  many  situations 
where  their  solutions  are  applicable,  the 
investigators  suggest  that  there  are 
situations  in  which  these  solutions  may  not 
apply  and  such  training  methods  inhibit  good 
judgment  in  these  situations. 

SET  encourages  the  development  of 
judgment  and  centers  training  around  three 
emergency  rules:  (a)  maintain  aircraft 
control,  (b)  analyze  the  situation  and  take 
proper  action,  and  (c)  land  as  soon  as 
practical.  The  underlying  concept  of  SET  is 
situational  training.  The  pilot  is  taught  to 
discriminate  between  relevant  and  irrelevant 
dimensions  of  situations  which  are 
systematically  manipulated  in  the  training 
program.  As  pointed  out  above,  this 
discrimination  process  is  fundamental  to  good 
judgment.  The  authors  suggest  a scenario 
development  program  using  instructor  training 
courses  as  one  of  the  major  sources  of  input. 

JUDGMENT  EVALUATION 

Perhaps  the  most  difficult  part  of  any 
study  of  human  judgment  is  the  evaluation  of 
performance.  The  reason  is  that  much  of  what 
must  be  evaluated  cannot  be  observed  directly 
but  must  be  inferred  from  observation  of  other 
related  behaviors.  From  discussions  with 
flight  instructors  and  pilot  examining 
personnel,  it  is  clear  that  judgment  is  not 
being  evaluated  effectively  today  (Jensen  and 
Benel , 1977). 

Although  flight  test  guides  published  by 
the  FAA  specify  that  civilian  pilots  are  to  be 
evaluated  for  their  "judgment"  capabilities, 


no  definition  of  Judgment  is  provided.  For 
this  evaluation,  examiners  primarily  depend 
upon  the  Judgment  of  flight  instructors  who 
have  the  opportunity  to  examine  their 
student's  decision-making  capability  over  a 
greater  variety  of  circumstances.  However,  in 
interviews  with  flight  instructors,  only  one 
was  found  who  admitted  to  having  failed  a 
student  purely  on  the  basis  of  poor  Judgment. 
Although  many  said  that  they  could  recognize 
poor  Judgment,  students  were  failed  on  the 
basis  of  a borderline  performance  of  some 
other  more  clearly  defined  flying  maneuver 
involving  skilled  performance. 

Some  ideas  for  Judgment  evaluation  are 
offered  by  Van  Dam  (in  Jensen  and  Benel, 
1977).  In  his  approach,  the  evaluation  begins 
with  psychological  and  intellegence  testing 
prior  to  admitting  students  for  flight 
instruction.  Initial  impressions  from  these 
pretraining  examinations  are  augmented  with 
other  subjective  indicators  of  judgment  such 
as  "obvious  effort  and  attention  to 
instruction,"  "relaxation,"  "division  of 
attention,"  "response  delays,"  "confidence," 
"capacity  for  problen-solving,"  and  "initia- 
tive." In  later  pilot  training,  evidence  of 
judgment  development  is  seen  through  an 

"eagerness  to  learn  or  high  motivation," 
"teachability,"  "adaptability  and  flexibi- 
lity," "an  intuitive  quality  in  thinking  or 
decision-making,"  "a  pattern  of  good  choices," 
and  "application  of  margins  and  allowances." 

As  indicated  in  the  training  system  model 
shown  in  Figure  2,  a vital  part  of  any 
educational  system  is  an  effective  evaluation 
program.  Training  must  be  continuously 

modified  in  response  to  the  results  of  these 
evaluations  and  progress  of  the  individual 
students  is  noted. 

The  requirements  of  pilot  judgmental 

evaluation  are  even  broader  than  these. 
Society  expects  pilots  to  make  decisions  based 
on  the  interests  of  passengers  and  property 
owners.  Therefore,  judgment  must  also  be 
evaluated  in  an  absolute  sense  against  this 
poorly  defined  scale. 

There  are  three  major  dimensions  along 
which  judgment  should  be  evaluated;  each 
presents  a unique  problem  to  the  evaluator: 

1.  The  assessment  of  judgmental  capabilities 
and  tendencies  prior  to  flight  training. 

2.  The  assessment  of  the  effects  of  training 
on  pilot  judgment. 

3.  The  assessment  of  the  amount  of  training 
transferred  to  the  operational  flying 
environment . 


Pratralalag  Evaluation 

It  is  important  from  the  standpoints  of 
both  safety  and  economics  to  identify  persons, 
prior  to  flight  instruction,  who  may  have 
difficulty  with  some  aspects  of  flying 
Judgment.  If  such  individuals  could  be 
identified,  they  could  either  be  discouraged 
from  seeking  flight  training  or  their  training 
programs  could  be  modified  to  offset  this 
deficiency. 

Unfortunately,  on  the  basis  of 
psychological  testing  research  to  date,  the 
predicted  success  of  such  a pretraining 
evaluation  program  is  not  very  good.  For 
example,  psychologists  and  others  have  made 
many  attempts,  with  little  success,  to 
identify  a general  personality  trait  known  as 
risk-taking  and  to  link  this  trait  to  accident 
proneness  (Shealy,  197*0.  However,  Shealy 
found  that  if  one  were  to  limit  the  scope  of 
the  test  to  specific  situations,  such  as 
down-hill  skiing,  its  predictive  validity 
would  be  greatly  increased.  Therefore,  it 
would  seem  that  efforts  to  develop  pretraining 
pilot  Judgment  prediction  tests  should  not  be 
discouraged  by  the  limited  success  of  the 
general  tests.  Instead,  efforts  should  be 
made  to  design  an  aviation  specific  test  with 
with  judgment  predictive  validity. 

Pretraining  evaluations  of  judgment 
ability  in  pilot  training  candidates  is  a 
potentially  useful  adjunct  to  the  entire 
training  and  evaluation  process.  Results  from 
such  test3  could  be  used  by  training 
management  to  adapt  their  programs  to 
emphasize  training  in  areas  identified  as 
potentially  weak  in  these  tests.  Flight 
instructors  could  be  alerted  to  possible 
weaknesses  in  Individual  students  and  adapt 
their  training  accordingly. 

Tests  which  could  identify  risk-taking 
tendencies  (Kogan  and  Wallach,  196*1;  Taylor 
and  Dunnette,  197*1)  and  tests  which  indentify 
accident  proneness  (Shaw  and  Sichel,  1971)  are 
potentially  useful  in  this  regard.  Situation 
specific  tests,  as  mentioned  above,  would  be 
useful  in  this  application  and  for  test 
development  for  use  in  later  training  as  well. 

Training  Evaluation 

The  second  major  dimension  along  which 
pilot  Judgment  must  be  evaluated  is  an 
assessment  of  the  amount  of  change  in  the 
pilot's  Judgment  performance  that  is  the 
result  of  training.  This  measure  provides  an 
indication  of  the  value  of  the  training 
program  as  well  as  indications  of  individual 
student  progress. 

The  development  of  clearly  defined 
judgmental  evaluation  criteria  presents  the 
greatest  challenge  to  effective  evaluation  of 


pilot  Judgment  in  all  phases  of  pilot 
training.  To  Insure  that  evaluations  are  made 
along  the  same  dimensions  as  the  training 
conducted,  the  development  of  these  criteria 
should  be  based  on  pre-established  behavioral 
objectives.  Judgmental  criteria  should 
consist  of  positive  statements  of  acceptable 
pilot  Judgmental  behavior  for  each  major  area 
of  flight  activity.  Similar  criteria  could  be 
developed  for  every  major  maneuver  taught. 
These  could  be  graded  by  the  instructor 
together  with  evaluations  of  knowledge  and 
skill  each  time  the  maneuvers  are  attempted. 

In  pilot  training,  for  each  level  of 
pilot  experience,  certain  judgmental  hurdles 
(proficiency  levels)  could  be  objectively 
specified.  The  instructor,  or  examiner  who 
evaluates  the  Judgme-is  would  have  a range  of 
acceptable  performances,  also  objectively 
specified.  Evaluation  of  pilot  Judgment  would 
be  a matter  of  comparing  performance  against 
the  established  criteria  in  carefully 
structured  situations. 

The  critical  point  for  Judgmental 
evaluation  in  a national  system  is  the  use  of 
the  same  criteria  by  ail  Judges  a3  well  as  by 
the  pilots  themselves.  One  way  to  insure 
standardization  of  Judgmental  evaluations  is 
to  use  a nationwide  CAI  system  to  administer 
tests  at  specific  times  during  each  student's 
training  program.  Results  of  such  tests  could 
be  used  to  modify  the  individual  student's 
training  or  the  training  program  as  a whole. 

Transfer  Evaluation 

The  final  dimension  along  which  pilot 
judgment  must  be  evaluated  is  an  assessment  of 
the  amount  of  training  that  is  transferred  to 
the  operational  flying  environment.  This 
means  that  students  who  have  received  special 
Judgmental  training  are  compared  with  those 
who  have  not  received  such  training  after  both 
groups  have  moved  into  the  operational  flying 
environment.  The  results  of  this  evaluation 
are  used  to  modify  both  student  selection 
criteria  (or  pretraining  examinations)  and 
program  need  assessments. 

The  criteria  for  this  evaluation  are 
basically  the  same  as  those  used  in  training 
evaluations  except  that  they  are  more  highly 
influenced  by  societal  demands.  Measures  of 
judgmental  training  transferred  can  be  made  in 
terms  of  the  number  of  accidents  or  incidents 
due  to  faulty  pilot  Judgment  reported  within 
the  respective  groups. 

Qpsna.UQaa.Uz log  Judgmental  Evaluations 

The  definition  of  pilot  judgment  ha3  two 
components:  discrimination  among  situational 
dimensions  and  response  selection.  Both 
components  must  be  evaluated.  To  operational- 
ize these  components  for  use  in  any  specific 
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training  or  testing  situation,  the  evaluator 
may  ask  the  following  questions: 


Training  Analysis  and  Evaluation  Group, 
NTEC,  1972. 


1.  £an  discriminative  judgment : Did  the  stu- 
dent consider  all  of  the  alternatives 
available  to  him?  Did  he  consider  all  of 
the  relevant  information  and  assign  proper 
weights  to  each  item?  Did  he  integrate 
the  relevant  information  efficiently 
before  making  his  choice? 

2.  For  response  selection  tendencies:  Did  the 
student  exhibit  any  tendency  to  consider 
factors  other  than  safety  (such  as  his  own 
self-esteem,  adventure,  or  social  pres- 
sure) in  making  his  response  selection? 
Did  he  seem  to  be  highly  prone  to  use 
semi-relevant  factors,  such  as  financial 
gain  or  convenience,  in  situations  where 
safety  should  have  been  the  primary 
consideration? 

The  use  of  such  criteria  as  these 
requires  more  of  the  evaluator  than  just  an 
occasional  passing  glance  at  the  instrument 
panel.  It  requires  the  careful  structuring  of 
the  situation,  perhaps  hypothetically,  and  a 
careful  examination  of  actions  taken  by  the 
student.  It  probably  would  require  a dialogue 
between  the  student  and  the  evaluator  to 
establish  what  the  student  actually  considered 
in  making  his  choice.  Each  evaluation  must  be 
considered  a training  device  as  well,  and  as 
such,  feedback  should  be  given  to  the  student 
concerning  all  aspects  of  the  decision 
situation  known  to  the  evaluator.  It  is 

recognized  that  evaluations  of  this  sort  place 
high  demands  on  the  flight  instructor. 

Nevertheless,  they  seem  to  be  warranted  in 
view  of  the  high  number  of  fatalities  caused 
by  faulty  judgment,  a factor  that  is  hardly 
being  evaluated  at  all  under  the  present 

system. 
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INTRODUCTION 

Whether  in  an  aircraft  or  flight  simu- 
lator, pilots  use  information  from  a variety 
of  sensory  mechanisms  to  determine  their  esti- 
mate of  orientation  and  motion.  An  under- 
standing of  this  process  and  a quantitative 
model  are  essential  for  development  of  effec- 
tive simulator  motion  cueing  devices.  A multi- 
sensory  model  for  dynamic  spatial  orientation 
is  being  developed  for  this  purpose  and  will 
be  used  by  the  Air  Force  as  a data  base  for 
development  of  current  and  future  force  simu- 
lation devices.  The  model  is  a potential 
tool  for  objectively  gauging  the  relative 
fidelity  of  different  simulation  strategies 
and  the  relative  importance  of  different  cue- 
ing devices  under  various  conditions. 

Aircraft  or  simulator  motion  is  trans- 
lated into  stimuli  which  are  processed  by 
dynamic  models  of  the  appropriate  sensors 
(visual,  vestibular,  tactile,  and  propriocep- 
tive), and  are  then  fed  to  a central  esti- 
mator which  has  been  modeled  as  a linear 
optimal  estimator,  specifically  a steady 
state  Kalman  Filter.  In  addition  to  the 
linear  estimation  process,  some  nonlinear 
effects,  such  as  the  well  documented  delay 
in  onset  of  visually  induced  motion,  require 
nonlinear  additions  to  the  model.  We  have 
attempted  to  keep  such  additions  to  a mini- 
mum so  as  to  retain  the  uniqueness  and 
conceptual  appeal  of  a linear  optimization 
algorithm. 

The  model  has  been  implemented  as  a 
computer  program  and  has  predicted  some  of 
the  important  qualitative  characteristics 
of  human  dynamic  spatial  orientation  under 
combined  wide  field  visual  motion  and  plat- 
form motion.  Several  types  of  special 
tactile  and  proprioceptive  cues  are  also 
being  considered  but  have  not  been  validated. 

The  modeling  effort  has  underscored 
the  need  for  additional  data  in  some  areas 
and  several  experiments  have  been  suggested 
to  help  fill  these  gaps. 


SENSORS  INCLUDED  IN  THE  MULTISENSORY  MODEL 

The  vestibular  sensors,  located  in  the 
labyrinthine  structure  behind  the  auditory 
portion  of  each  ear,  are  the  most  thorough- 
ly studied  and  well  defined  of  the  sensory 
systems  under  consideration.  The  semi- 
circular canals  are  the  rotation  sensing 
component  of  the  vestibular  system  and 
respond  to  angular  acceleration  as  would  a 
heavily  damped  torsion  pendulum  with  some 
additional  rate  sensitivity  and  adapta- 
tion. 1234  For  modeling  purposes,  the 
two  sets  of  canals  have  been  replaced  by  a 
single  cyclopian  set  at  the  center  of  the 
head  and  are  modeled  as  shown  in  Figures 
1 and  2. 
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Figure  1.  Cyclopian  Sensor  Coordinates 


ACCELERATION  CANAL  MECHANICAL  * 

STIMULUS  t AFFERENT  DYNAMICS  THRESHOLD  1 

a«c 

a»e 

a*c 

0 574 » i »»I00) 

1 

J_r 

*■ 

(•  tO  l ) ( • ♦ 0 033 ) 

» 

AFFERENT 

FIRING 

RATE 


Figure  2.  Semicircular  Canal  Model 
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Figure  3.  Otolith  Model 


The  otoliths,  which  form  the  other  com- 
ponent of  the  vestibular  system,  sense  grav- 
ito-inertial  force  much  like  accelerometers. 
As  shown  in  Figure  3,  they  are  modeled  as 
mechanical  accelerometers  with  some  addition- 
al rate  sensitivity  presumably  due  to  affer- 
ent processing.5  6 Once  again,  a cyclopian 
system  has  been  used  for  modeling  purposes 
and  is  assumed  to  be  located  at  the  center 
of  the  head.  (See  Figure  1). 


It  has  long  been  observed  that  moving 
visual  fields  can  induce  a sensation  of 
motion  such  as  that  sometimes  experienced 
when  a neighboring  train  in  a railway  sta- 
tion begins  to  pull  away.  It  has  been  shown 
that  visual  self-motion  sensation  is  propor- 
tional to  the  velocity  of  the  background  per- 
ipheral vision  field  up  to  a saturation  level 
and  that  the  effectiveness  of  the  stimulus 
is  related  to  the  spatial  frequency,  contrast 
and  resolution  of  elements  in  the 
field.7  0910  The  current  model  considers 
only  out-the-window  peripheral  view  fields 
such  as  moving  clouds  or  star  patterns,  and 
specifically  excludes  cockpit  instrument 
readings  or  structures  such  as  recognizable 
landmarks  or  ar,  horizon  reference. 


Although  movement  in  the  peripheral 
visual  field  may  not  immediately  cause  a 
sensation  of  self-motion,  the  field  motion  is 
detected  almost  immediately  by  the  eye  after 
only  a short  neural  transmission  delay.  Dy- 
namics of  the  visual  sensors  have,  therefore, 
been  modeled  as  unity,  and  dynamics  associ- 
ated with  onset  of  the  self-motion  sensation 
are  ascribed  to  higher  centers. 

Tactile  and  proprioceptive  contribu- 
tions to  motion  perception  are  not  nearly 
as  well  defined  as  are  the  visual  and  vesti- 
bular elements.  Figure  4 shows  a simple 
tactile  receptor  model  proposed  as  an  initial 
attempt  to  consider  "seat-of-the-pants" 


sensation  during  aircraft  or  simulator 
flight.  The  model  uses  a single  "lead-lag" 
transfer  function  to  represent  the  major 
characteristics  of  three  distinct  mechano- 
receptors:  rapidly  adapting  Pacinian  cor- 
puscles usually  found  in  subcutaneous 
tissue,  and  more  slowly  adaptinq  type  I and 
type  II  cutaneous  receptors.11  17  ” 

Several  such  tactile  model  elements  are  used 
to  consider  forces  applied  to  various  areas 
of  the  body. 

Although  proprioceptive  sensation  in- 
volves numerous  muscle  length,  muscle  ten- 
sion, and  joint  position  receptors,  the 
initial  multi  sensory  model  considers  only 
muscle  spindle  (length  sensor)  response 
to  lateral  gravito-inertial  force  on  the 
head-neck  system.  The  head-neck  system, 
which  may  be  thought  of  as  an  inverted  pen- 
dulum that  is  balanced  on  the  body  trunk 
with  the  aid  of  neck  muscles,  provides  a 
very  important  spatial  orientation  cue  and, 
among  applicable  proprioceptive  mechanisms, 
lends  itself  to  the  most  straight-forward 
modeling.  Figure  5,  adapted  from  Gum14, 
represents  an  initial  attempt  to  include 
proprioception  in  the  multisensory  model 
and  will  probably  be  refined  and  expanded 
as  more  experimental  data  becomes  avail- 
able. 
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Figure  4.  Tactile  Model 
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Figure  5.  Head-Neck  Proprioception 
Model  (after  Gum*14) 
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CENTRAL  PROCESSOR 

The  central  processor,  representing  cen- 
tral nervous  system  function,  is  modeled  as 
an  optimal  estimator  which  weights  each  in- 
formation channel  according  to  certain 
a priori  assumptions.  To  assign  optimal 
weights,  the  processor  must  have  knowledge  of 
the  sensor  dynamics,  the  expected  stimulus 
spectrum,  and  the  noise  in  each  sensor 
measurement.  This  set  of  assumptions  is  re- 
ferred to  as  the  Internal  Model  since  it  rep- 
resents the  system's  knowledge  about  itself. 
If  these  assumptions  include  linearity, 
stationarity,  and  white  Gaussian  noise 
processes,  the  optimal  linear  estimator  re- 
duces to  the  well  known  steady  state  Kalman 
Filter  and  weights  or  "Kalman  gains"  can  be 
chosen  to  minimize  rms  error  of  the  estimate. 
Although  the  biological  system  is  certainly 
far  more  complex,  our  approach  has  been  to 
start  with  relatively  straight-forward  Kalman 
Filter  blending  and  increase  the  model  com- 
plexity only  as  it  proves  necessary. 

Figure  6 shows  a schematic  view  of  the 
multi -sensory  model  employing  a linear  filter 
alone  as  central  processor.  After  optimal 
gains  are  set,  deterministic  stimuli  corres- 
ponding to  aircraft  or  simulator  motion  are 
processed  by  individual  sensor  models  (which 
may  include  nonlinearities)  and  fed  to  the 
steady  state  Kalman  Filter.  The  filter,  in 
turn,  estimates  state  variables  corresponding 
to  perception  of  inertial  state  and  spatial 
orientation. 

Since  the  proper  or  logical  values  for 
some  Internal  Model  parameters  such  as  the 
input  and  measurement  noise  spectra  are 


INTERNAL  MODEL 


Figure  6.  Multisensory  Model  Using 
Steady  State  Kalman  Filter  To 
Represent  Neural  Central  Processing 


essentially  unknown,  there  is  considerable 
leeway  for  adjusting  model  response  within 
the  framework  described.  These  unknown 
parameters  have  been  "tuned"  to  bring  model 
responses  closer  and  closer  to  known  human 
response,  and  we  feel  that  further  tuning 
will  result  in  additional  improvement.  The 
initial  model  represents  a naive  subject  who 
has  no  advance  knowledge  of  the  stimulus  to 
be  received  or  of  the  limits  of  his  vehicle. 
The  active  pilot  with  varying  skill  levels 
and  operating  under  different  workloads  is 
considered  to  be  an  extension  of  the  more 
basic  naive  case.  Possible  means  for 
extending  the  model  to  the  active  pilot  are 
discussed  later  on. 

MODEL  TIME  RESPONSE 

The  model  described  in  the  previous 
section  has  been  implemented  on  a computer 
using  the  linear  estimator  alone.  Figure  7 
shows  several  responses  to  combined  visual 
field  and  platform  rotation  about  the  yaw 
axis.  When  no  visual  information  is  present 
(in  dense  fog  or  in  the  dark,  etc.),  the 
model  angular  velocity  estimate  (curve 
labeled  RD)  shows  the  well  documented  adapta- 
tion to  continuous  rotation  which  proved 
to  be  such  a problem  for  pre-instrument  era 
pilots.  It  is  also  significant  that  the 
decay  in  angular  velocity  perception  lags 
behind  that  of  the  semicircular  canal  affer- 
ent response  since  this  relation  has  also 
been  observed  experimentally. 


KALMAN  FILTER  MOOEL 
RESPONSES  TO  YAW  VELOCITY  ‘STEP’ 

r STIMULUS 


Figure  7. 

Model  Response  To  Combined  Platform 
and  Wide  Visual  Field  Vaw  Motion 


When  wide  visual  field  information  is 
present  and  is  consistent  with  a physically 
stable  visual  surround,  angular  velocity 
perception  is  a fairly  accurate  reflection 
of  true  angular  velocity  as  shown  by  the 
curve  labeled  RL.  When  the  visual  surround 
is  fixed  to  the  rotating  platform  so  that 
the  visual  system  always  reports  zero 
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velocity,  the  curve  labeled  RFF  shows  an 
adapting  response  similar  to  the  case  of 
rotation  in  the  dark  (RD),  but  with  a 
smaller  magnitude  and  faster  adaptation  con- 
stant. The  RFF  curve  makes  intuitive  sense 
since  the  visual  surround  is  actively  "deny- 
ing" the  presence  of  motion,  not  just  failing 
to  provide  information.  In  most  cases,  how- 
ever, experimental  observations  have  not  pro- 
vided enough  evidence  to  clearly  distinguish 
between  the  dark  (RD)  and  vehicle  fixed- 
field  (RFF)  situations,  and  in  some  cases 
RFF  appears  to  be  the  stronger  response, 
especially  for  very  low  accelerations. 

When  the  platform  is  stationary  and  the 
visual  surround  rotates  as,  for  instance,  in 
a fixed-base  simulator  with  a wide  field 
visual  display,  the  curve  labeled  C V shows  a 
gradual  onset  of  angular  velocity  sensation 
over  a period  of  about  ten  seconds.  This  is 
a classical  circularvection  (visually  induced 
rotation)  response  with  one  qualification. 

The  exponential  onset  of  sensation  is  shown 
to  begin  immediately  after  the  onset  of 
visual  field  motion  whereas  a delay  is  often 
observed  in  human  response.  The  onset  delay 
appears  to  be  a highly  variable  and  non- 
linear phenomenon  which  cannot  be  produced 
by  the  steady  state  Kalman  Filter  alone  and 
requires  the  addition  of  a nonlinear  model 
element  as  described  in  the  next  section. 

Except  for  the  absence  of  a circular- 
vection onset  delay,  the  relations  between 
the  various  curves  of  Figure  7 are  quite 
similar  to  those  observed  experimentally. 
Exercise  of  the  Kalman  Filter  model  during 
pitch  and  roll  plane  motion  has  also  pro- 
duced several  important  response  character- 
istics observed  in  human  spatial  orientation 
during  combined  platform  and  visual  field 
motion.  Among  these  predictions  are  the 
following: 

1.  Gradual  illusion  of  pitch  up  during 
prolonged  or  very  large  forward  acceler- 
ation such  as  commonly  experienced  by 
aircraft  carrier  pilots  during  catapult 
launch. 15 


Figure  8.  Model  Responses  to  Forward 
Acceleration  Stimuli  showing 
Characteristic  Pitch-Up 
Illusion 


Figure  9.  Model  Response  to  a Pitch-Up 

Stimulus  with  Confirming  Visual 
Cues.  Since  the  model  does  not 
consider  specific  structure  in 
the  visual  surround  such  as  an 
horizon  reference,  vision  pro- 
vides only  pitch  /elocity  and 
not  static  pitch  ingle  informa- 
tion. 


2.  Fairly  accurate  perception  of  for- 
ward acceleration  in  the  presence  of 
confirming  visual  cues. 

3.  Fairly  accurate  perception  of  roll 
and  pitch  orientation  changes  so  long  as 
all  cues  are  consistent  (i.e.,  no  trans- 
lator accelerations  or  contradictory 
visual  stimuli). 


Preliminary  exercise  of  tactile  and 
proprioceptive  model  components  has  yielded 
responses  which  appear  to  be  quite  reason- 
able (Figure  8 contains  an  example  of  com- 
bined vestibular-tactile  response),  but 
truly  meaningful  evaluation  of  these  com- 
ponents requires  supportive  psychophysical 
data  as  yet  unavailable. 


4.  Gradual  acceptance  of  visual  field 
translator  velocity  (1  inearvection) 
when  there  is  no  platform  motion.9 

5.  Static  tilt  illusion  accompanying 


Quantitative  characteristics  of  the 
model  responses  can  be  further  adjusted  by 
additional  "tuning"  of  the  linear  filter. 
For  example,  the  RD  curve  shown  in  Figure  7 
(yaw  velocity  perception  in  the  dark) 
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probably  lags  too  far  behind  the  canal  affer- 
ent response  curve  and  should  cross  zero  ten 
or  fifteen  seconds  earlier.  Some  of  the 
responses  described  are  also  accompanied  by 
secondary  responses  which  need  to  be  explored 
further.  For  instance,  the  pitch-up  sensa- 
tion of  Figure  9 is  shown  to  be  accompanied 
by  a relatively  small  but  unexpected  trans- 
lator motion  sensation. 

Notwithstanding  the  need  for  further 
tuning,  the  linear  estimator  alone  is  in 
significant  agreement  with  a wide  range  of 
known  human  response  characteristics. 
Additions  to  the  central  processor  model, 
although  representing  a slight  retreat  from 
the  conceptual  appeal  of  a pure  optimal 
estimator,  can  be  used  to  produce  some  of 
the  nonlinear  human  responses  that  the 
Kalman  Filter  alone  is  incapable  of  generat- 
ing. 

NONLINEAR  ADDITION'  TO  THE  MODEL 


Onset  of  visually  induced  motion  sensa- 
tion has  been  observed  to  occur  only  after  a 
highly  variable  delay  which  is  a function  of 
mental  set,  compelling  nature  of  the  display, 
and  stimulus  magnitude.  For  modeling  pur- 
poses, both  a favorable  mental  set  and  a 
compelling  display  are  assumed.  Given  these 
conditions,  the  onset  delay  is  often  undetect- 
able when  visual  stimulus  magnitudes  are  near 
vestibular  thresholds  and  becomes  larger  as 
the  visual  stimulus  exceeds  these  values. 

The  onset  delay  also  vanishes  in  the  presence 
of  confirming  vestibular  cues  of  sufficient 
magnitude. 

Figure  10  shows  a generalized  cue  con- 
flict scheme  adapted  for  use  with  the  Kalman 
Filter  estimator  from  a conflict  model  dev- 
eloped by  Zacharias  and  Young.17  Visual 
information  is  used  to  estimate  expected 
vestibular  response  and  then  compared  to 
actual  vestibular  response.  The  resultant 
error  signal  is  rectified  and  passed  through 
an  adaptation  operator  so  that  any  sustained 


rxn  , 

•rhl 

_ J 


Figure  10.  Conflict  nonlinearity  combined 
with  Kalman  Filter  Model 


errors  will  eventually  be  ’‘washed-out."  A 
switch  keeps  visual  gain  (K)  at  unity  so 
long  as  the  rectified,  "washed-out"  error 
is  small,  and  gradually  drops  the  gain  to 
zero  as  this  error  approaches  vestibular 
threshold  value  (e). 

Figure  11  shows  the  response  of  the 
combined  linear  central  processor  and  cue 
conflict  element  to  a series  of  steps  in 
visual  field  yaw  velocity,  all  beginning 
at  t=0.  Responses  now  show  an  onset  delay 
which  increases  with  stimulus  magnitude  as 
found  experimentally.  The  presence  of  the 
conflict  nonlinearity  affects  only  the 
circularvection  (CV)  response  in  Figure  7 
When  both  the  visual  and  vestibular  systems 
are  stimulated  (RL  curve  of  Figure  7),  the 
conflict  is  zero  and  visual  gain  (K)  remains 
unity.  In  the  other  two  stimulus  combina- 
tions presented  in  Figure  7,  the  visual 
signal  is  either  zero  (rotation  with  a 
vehicle-fixed  visual  surround)  or  nonexis- 
tent (rotation  in  the  dark)  and  K is,  there- 
fore, irrelevant. 


Figure  11.  Circularvection  step  response 
of  model  with  cue  conflict 
element  included.  Stimuli  are 
1.5°/sec,  5°/sec,  and  15°/sec 
steps  in  visual  field  yaw 
velocity  beginning  at  t=0. 

Certain  illusory  effects  associated 
with  static  orientation  in  different  spec- 
ific force  environments  have  long  been 
recognized  and  classified  as  Aubert,  Meuller 
and  Elevator  illusion  effects.  The  Aubert 
effect  describes  a typical  underestimation 
of  tilt  angle  in  a lg  or  less  than  lg 
environment;  the  Meuller  effect  refers  to  a 
characteristic  overestimation  of  tilt  angle 
in  a greater  than  lg  environment;  and 
Elevator  illusion  is  a pitch-up  sensation 
that  often  accompanies  increased  g2. 

Ormsby3  has  shown  that  the  above  effects 
can  be  generated  by  assuming  that  the 
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lonlinear  function  of  Figure  12  operates  on 
saccular  otolith  information  ("zo"  axis  of 
Figure  1).  The  mul  tisensory  model  must  in- 
corporate a similar  nonlinear  element  if  it 
is  to  exhibit  Aubert,  Meuller,  and  Elevator 
illusions. 
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Figure  12.  Saccule  Nonlinearity  (redrawn 
from  Ormsby3). 


As  the  model  is  exercised  more  thorough- 
ly and  as  more  experimental  data  becomes 
available,  it  may  be  necessary  to  further 
increase  the  complexity  of  the  central  pro- 
cessor model  in  order  to  obtain  a more 
precise  match  with  certain  observed  human 
responses.  Our  philosophy,  however,  is  to 
retain  the  optimal  estimator  concept  to  the 
greatest  degree  possible. 

ACTIVE  PILOT 


Perception  is  dramatically  affected  by 
mental  set,  and  prior  knowledge  or  expectancy 
of  motion  will  certainly  influence  motion 
perception.  If  the  multisensory  model  is  to 
represent  an  active  pilot  as  opposed  to  a 
naive  subject,  it  must  be  extended  to  con- 
sider the  predictive  information  available 
to  the  pilot.  Some  ideas  for  such  extension 
are  presented  in  this  section. 

A skilled  pilot  may  be  presumed  to  have 
an  accurate  expectation  of  input  spectra 
associated  with  the  maneuver  being  initiated. 
One  modeling  approach,  therefore,  is  to 
assume  that  the  central  processor  has  correct 
knowledge  of  the  input  spectrum  for  a specif- 
ic maneuver  or  portion  of  a maneuver.  The  Kal- 
man gains  to  be  used  during  this  maneuver 
segment  can  be  calculated  using  that  spec- 
ific spectrum.  Since  a changing  input  spec- 
trum does  contradict  the  steady-state 
assumption,  it  might  be  necessary  to  use  a 
time  varying  Kalman  filter  in  place  of  the 


steady-state  version;  preliminary  results 
with  the  above  technique  suggests  that  the 
steady-state  filter  may  indeed  be  inadequate. 
Implementation  of  a time  varying  filter  is 
well  known  and  certainly  feasible  although 
far  more  cumbersome  in  terms  of  programing 
complexity  and  computer  time  requirements. 

A second  approach  is  to  incorporate 
control  stick  position  as  an  additional  in- 
put to  the  Kalman  Filter  (Figure  13).  A 
skilled  pilot  knows  that  a given  adjustment 
of  aircraft  controls  results  in  a certain 
change  in  aircraft  state,  and  thus  know- 
ledge of  control  input  becomes  data  for  the 
central  processor.  A pilot  with  less  skill 
might  have  less  confidence  in  the  stick 
position  measure,  as  reflected  in  a larger 
expected  measurement  noise  (Vj^.^^),  or  may 
rely  on  a less  accurate  intern,.,  „iodel  of 
aircraft  dyna*  ' , the  rilot  is  extreme- 

ly busy,  si  . . > pres:.'  ably  siphon 
nervous  system  attention  . rom  sensor  motion 
cue  signals,  .ng  the  system  less  sensi- 
tive to  these  signals.  The  decreased 
sensitivity  can  be  expressed  in  the  Kalman 
Filter  model  by  increasing  the  expected 
sensory  measurement  noise  values. 


, 


Figure  13.  Model  Extension  to  Include 
Active  Pilot 

APPLICATION 

An  effort  is  currently  underway  at  the 
Air  Force  Human  Resources  Laboratory  which 
is  aimed  at  determining  motion  cue  require- 
ments of  fighter  aircraft  simulators.  The 
multisensory  model  will  be  an  integral 
part  of  this  effort  with  eventual  applica- 
tion both  to  hardware  and  software  develop- 
ment. For  example,  on  a maneuver- by- 
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maneuver  basis,  the  model  can  be  used  to 
compare  perceived  motions  and  forces  created 
by  an  aircraft  with  those  created  by  simu- 
lator cueing  hardware  and  drive  algorithms. 
Potential  simulator  deficiencies  may  thus 
be  spotted  prior  to  extensive  subjective 
testing  and  sources  of  existing  deficiencies, 
causing  poor  pilot  performance  or  complaints 
that  the  simulation  "doesn't  feel  right," 
may  be  identified.  It  is  anticipated  that 
the  model  will  eventually  help  to  address 
questions  concerning  the  optimum  and  most 
cost-effective  combinations  of  platform 
motion,  visual  scene  motion,  g seat  stimu- 
lation, etc. 

ADDITIONAL  DATA  REQUIREMENTS 

The  required  data  concerning  receptor 
physiology  and  the  results  of  sensory  inter- 
action is  by  no  means  complete.  Additional 
data  in  the  following  areas  would  be  extreme- 
ly helpful  to  modeling  efforts: 

1.  Data  needed  to  help  validate  or 

modify  the  central  processor  model 

...Psychophysical  data  is  needed  to 
determine  human  perception  of  sev- 
eral motion  cue  situations  that 
commonly  occur  in  aircraft  but 
rarely  elsewhere.  It  would  be 
extremely  useful  to  have  data  con- 
cerning orientation  and  angular 
velocity  perception  during  coordi- 
nated maneuvers  (maneuvers  in  which 
angular  velocity  is  not  accompanied 
by  a confirming  change  in  specific 
force  direction). 

...Experiments  are  needed  to  clarify 
the  situations  under  which  tactile 
stimuli  are  interpreted  as  motion 
or  orientation  cues.  In  particular, 
subjective  magnitude  estimates  of 
acceleration  during  different  com- 
binations of  g-cueing  seat  stimula- 
tion and  real  platform  motion  would 
be  very  helpful.  Such  data  would 
be  directly  applicable  to  g-cueing 
seat  development  as  well  as  to  the 
modeling  effort. 

...In  the  area  of  visually  induced 
motion,  data  is  needed  to  determine 
the  amount  of  confirming  platform 
motion  and/or  tactile  stimulation 
required  to  produce  immediate  and 
sustained  visually  induced  motion 
sensations.  In  addition  to  its  use 
for  model  validation,  this  data 
would  be  directly  applicable  to 
specification  of  platform  motion 
requirements  in  wide  field  visual 
simulators. 

...Most  motion  threshold  figures 

currently  available  represent  optimal 
detection  conditions  and  may  be 


unrealistic  for  application  to  a 
busy  pilot  performing  demanding 
tasks  in  a noisy  aircraft  or  sim- 
ulator. Experiments  are  needed  to 
clarify  the  effect  of  workload, 
pilot  skill,  and  masking  vibration 
so  that  effective  thresholds  can  be 
properly  incorporated  in  perception 
models  and  applied  to  simulation 
problems. 

2.  Data  needed  to  validate  or  modify 

individual  sensor  models 

...There  is  currently  very  little  data 
available  to  support  the  head-neck 
proprioception  model  described  in 
this  paper.  Measurements  of  dynamic 
head  displacement  under  lateral 
and  longitudinal  gravito-inertial 
force,  as  well  as  electro-physiolog- 
ical measurements  of  head-neck 
system  response  during  such  stimuli 
would  be  a great  aid. 

...The  tactile  cue  processing  model 
currently  employed  is  based 
primarily  on  data  generated  from 
neural  recordings  or  vibro-tactile 
threshold  measures  in  which  small 
area  stimuli  were  applied  to  the 
forearm  or  hand.  Threshold  and 
sensitivity  to  broad  area  tactile 
stimulation  over  the  back  and 
buttocks  needs  to  be  investigated 
so  that  the  tactile  model  component 
can  be  improved  and  expanded. 

3.  Data  to  determine  the  proper  stimu- 
lus input  to  the  model 

...The  relation  between  aircraft  motion 
and  body-seat  pressure  currently  is 
not  well  known,  yet  is  very  impor  - 
tant  to  the  modeling  effort  since  it 
forms  the  input  to  the  tactile  com- 
ponent of  the  multisensory  model. 
Studies  of  dynamic  body-seat 
pressure  distribution  in  response 
to  changes  in  gravito-inertial 
force,  especially  during  typical 
aircraft  maneuvers,  will  be  a great 
aid  both  in  modeling  and  in  develop- 
ment of  g-cueing  seat  drive  algo- 
ri thms . 

Preparations  are  currently  underway 
for  three  experiments  that  will  address 
some  of  the  needs  listed  above.  It  has  been 
proposed  that  the  Air  Force  Total  In-Flight 
Simulator  (TIFS)  be  used  to  gather  psycho- 
physical motion  perception  data  and  to 
simultaneously  study  several  aspects  of  the 
motion  cue  environment  during  standardized 
maneuvers.  The  maneuvers,  which  include 
coordinated  turns,  "roller-coaster  motions," 
and  sustained  longitudinal  accelerations, 


are  designed  to  emphasize  stimuli  which  can- 
not be  adequately  represented  in  existing 
motion-based  simulators.  Measurements  will 
be  made  of  perceived  direction  of  vertical 
and  magnitude  estimates  of  roll -rate  and 
translatory  acceleration  using  a special 
indicator  and  employing  an  otherwise  passive 
subject.  The  dynamics  of  seat  pan  and  back- 
rest pressure  distribution  for  both  the 
pilot  and  passive  subject  will  be  measured 
with  a set  of  pressure  transducers  sewn  into 
or  laid  over  the  seats.  Video  tape  record- 
ings of  the  pilot's  and  passive  subject's 
head  positions  with  respect  to  the  cockpit 
will  be  made  for  later  analysis,  and  neck 
muscle  activity  may  also  be  measured  using 
EMG  electrodes.  Full  documentation  of  the 
aircraft's  specific  forces,  angular  rates, 
and  Euler  angles,  will  be  obtained  from  an 
on-board  inertial  system. 

The  NASA  Manned  Carrying  Rotation  Device 
(MCRD),  because  of  its  capacity  for  smooth 
rotation,  is  ideally  suited  for  study  of 
yaw  acceleration  threshold  and  time-to- 
detect  as  a function  of  masking  vibration 
frequency  and  amplitude.  Such  an  experiment 
has  been  planned  with  Mr.  John  Stewart  and 
implemented  at  NASA  Ames  Research  Center. 

Dr.  David  Quam,  working  with  the  Air 
Force  Human  Resources  Lab,  is  currently 
preparing  to  use  the  Large  Amplitude  Multi- 
mode  Aerospace  Research  System  (LAMARS)  for 
a study  of  subjective  yaw  velocity  under 
combined  platform  and  visual  field  motion. 
This  will  include  determination  of  minimum 
confirming  platform  acceleration  and  minimum 
platform  excursion  needed  to  produce  iirniedi- 
ate  and  sustained  acceptance  of  visual 
field  yaw  motion. 

CONCLUSION 

A multisensory  motion  perception  model 
has  been  developed  using  an  optimal  estima- 
tion approach  to  the  integration  of  multiple 
sensory  inputs.  Although  the  biological 
system  is  far  more  complex  than  the  linear 
estimator  used  in  the  model,  the  linear 
estimator  alone  has  proven  capable  of 
exhibiting  correct,  qualitative  human 
response  characteristics  to  a surprising 
degree.  Some  highly  nonlinear  character- 
istics such  as  the  well -documented  delay  in 
onset  of  visually  induced  motion  are  not 
produced  by  the  linear  central  processor 
but  can  be  generated  by  adding  nonlinear 
elements  to  the  central  processor  model. 

It  is  felt  that  additional  improve- 
ment and  extension  of  the  multisensory  model 
is  possible  with  further  development  work, 
including  representation  of  the  active  pilot; 
however,  additional  experimental  data  in 
certain  areas  is  badly  needed  to  support 


such  effort.  In  its  eventual  application, 
the  model  will  be  useful  in  objectively 
predicting  motion  cue  requirements  for 
ground  simulation,  and  developing  drive 
algorithms  to  produce  the  highest  simulator 
fidelity  under  given  hardware  limitations. 
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VISUAL  CUE  MANIPULATION  IN  A SIMULATED  AIR-TO- SURFACE 
WEAPONS  DELIVERY  TASK 

DR.  RONALD  G.  HUGHES,  MAJOR  JAY  PAULSEN,  JR.,  REBECCA  BROOKS, 
and  2LT  WILLIAM  JONES 

Air  Force  Human  Resources  Laboratory 
Flying  Training  Division 
Williams  Air  Force  Base,  Arizona 


ABSTRACT 

Experienced  pilots  with  no  prior 
air-to-surface  training  practiced  a 30 
degree  dive  bombing  task  in  the  T- 37 
cockpit  of  the  Advanced  Simulator  for  Pilot 
Training  (ASPT)  located  at  the  Air  Force 
Human  Resources  Laboratory,  Flying  Training 
Division,  Williams  AFB,  Arizona.  Use  of  a 
bomb  impact  predictor  cue  by  one  group  of 
subjects  produced  no  better  performance 
than  that  of  a second  group  which  practiced 
without  the  cue.  Abrupt  removal  of  the 
cue,  which  during  training  was  not  made 
contingent  upon  performance,  produced  a 
significant  disruption  of  performance  on 
the  bombing  task,  both  in  terms  of  accuracy 
and  in  terms  of  variability  of 
performance.  Best  performance  was  obtained 
by  a third  group  for  which  the  gunsight 
itself  was  initially  withheld  in  training. 
The  results  are  discussed  in  terms  of  the 
need  in  future  systems  for  more  active 
control  over  the  stimuli  controlling  flying 
performance  as  well  as  the  need  for 
research  into  strategies  for  making  changes 
in  the  pilots  environment  contingent  upon 
performance . 

INTRODUCTION 

A basic  assumption  of  adaptive 
training  is  that  a difficult  task  can  be 
learned  more  efficiently  if  it  is  presented 
throughout  training  at  a level  of 
difficulty  that  is  matched  to  the 
individual's  current  ability  to  perform  the 
task.  Traditional  approaches  have  sought 
to  control  task  difficulty  through  the 
manipulation  of  the  response  characteris- 
tics of  the  task,  e.g.,  Gaines,  (1967); 
Norman,  Lowes,  and  Matheny,  (1972). 

In  some  instances,  attempts  have  been 
made  to  modify  the  difficulty  of  a complex 
motor  skill  task  by  providing  augmented 
feedback  to  the  performer.  Lintern  (1977) 
and  Lintern  and  Roscoe  (1978),  for  example, 
showed  that  the  use  of  an  "off  course" 
augmented  cue  could  be  used  to  enhance  the 
landing  training  of  naive  students. 


In  the  present  study,  an  attempt  was 
made  to  reduce  the  difficulty  of  an  air-to- 
surface  weapons  delivery  task  in  one  case 
through  the  addition  of  a visual  bomb 
impact  predictor  cue  and  in  a second  case 
through  selective  introduction  of  a 
dominant  visual  cue  inherent  in  the  task 
itself.  In  neither  case,  was  the 
introduction  or  withdrawal  of  the  cue 
contingent  upon  changes  in  student 
performance.  The  results  of  the  study  are 
discussed  in  terms  of  the  need  in  future 
sytems  for  more  active  operator/instructor 
control  over  the  visual  environment  of  the 
learner  as  well  as  the  need  for  research 
dealing  with  the  adaptive  use  of  augmented 
cuing. 

METHOD 

Subjects 

Twenty-two  T-37  Instructor  Pilots 
(IP's)  assigned  to  Williams  AFB,  Arizona 
served  as  subjects.  No  subject  had 
previous  experience  with  air-to-surface 
weapons  delivery.  Their  flying  experience 
both  in  terms  of  T-37  flying  hours  as  well 
as  overall  military  flying  hours  is  given 
in  Table  1. 

TABLE  1 

FLYING  EXPERIENCE 


T-37  Hours  Total  Hours 


Group 

s.d. 

X 

s.d. 

Standard 

642.00 

338.93 

1580.32 

1185.86 

Predictor 

701.67 

297.21 

1150.17 

720.93 

No  Gunsight 

922.00 

168.88 

1158.00 

152.30 

Overall 

721.91 

306.86 

1367.02 

919.50 

Instructors  for  the  bombing  task  were 
Instructor  Pilots  assigned  to  AFHRL's 
Flying  Training  Division  at  Williams  AFB. 
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All  were  familiar  with  the  air-to-surface 
task  being  taught.  None  were  considered, 
however,  as  "TAC-qualified"  instructors. 
Because  of  constraints  on  the  use  of 
instructors,  no  attempt  was  made  to 
counterbalance  instructors  across 
conditions. 

Apparatus 


The  Advanced  Simulator  for  Pilot 
Training  (ASPT)  located  at  AFHRL/FT  was 
used  for  training  of  the  air-to-surface 
task.  Technical  references  for  this  device 
are  found  in  Gum,  Albery,  and  Basinger 
(1975)  and  in  Rust  (1975).  For  the  study, 
the  g-seat  was  inflated  but  not  otherwise 
operational.  Neither  was  the  motion 
platform  in  operation.  The  computer 
generated  visual  scene  was  presented  via 
ASPT's  seven  36-inch  monochromatic 
cathode-ray  tubes  placed  around  the  cockpit 
giving  the  pilot  +110  degrees  to  -40 
degrees  vertical  cuing  and  + 150  degrees  of 
horizontal  cueing.  Configuration  of  the 
visual  scene  for  this  study  included  a 
conventional  gunnery  range  visual  data  base 
similar  to  that  developed  for  project  2235 
and  that  used  by  Gray  and  Fuller  (1977)  as 
well  as  a depressible  bombing  sight  (A-37 
Optical  Sight  Unit).  The  aerodynamic 
mathematical  models  driving  the  simulator 
were  those  of  the  T-37  aircraft. 

The  predictor  cue  used  in  the  present 
study  consisted  of  a hexagonal-shaped  spot 
of  light,  approximately  30  feet  (9.114 
meters)  in  diameter,  which  appeared  from 
the  air  to  move  along  the  ground,  giving  a 
continuous  indication  to  the  pilot  of  where 
a bomb  would  impact  if  dropped  at  that 
point  in  time.  The  manner  in  which  the  cue 
was  generated  by  the  system  and  other 
details  of  its  implementation  are  described 
by  Cyrus,  Templeton,  and  McHugh  (in 
press).  The  cue  was  programmed  so  as  to  be 
available  under  command  of  the  console 
instructor.  In  the  present  study,  the  cue 
was  illuminated  continuously.  No  provision 
was  made  by  the  system  to  systematically 
vary  the  intensity  of  the  cue. 

Procedure 


Individuals  in  each  of  three  separate 
groups  (referred  to  hereafter  as  Standard 
Group,  Predictor  Group,  and  No-Gunsight 
Group)  of  T-37  Instructor  Pilots  (IPs) 
performed  15  repetitions  of  a 30  degree 
dive  bomb  task  in  the  T-37  cockpit  of  the 
Advanced  Simulator  for  Pilot  Training 
(ASPT).  Prior  to  entering  the  simulator, 
each  subject  completed  a short  paper  and 
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pencil  pretraining  exercise  intended  to 
familiarize  the  subject  with  the  basic 
elements  of  the  task.  Once  the  subject 
entered  the  simulator,  the  subject  was 
presented  a recorded  demonstration  of  a 30 
degree  dive  bomb  task.  For  the  predictor 
group,  the  demonstration  contained  the 
predictor  cue  in  addition  to  the  gunsight. 
For  the  no  gunsight  group,  the 
demonstration  contained  neither  the 
gunsight  nor  the  predictor  cue.  For  the 
standard  group,  the  demonstration  contained 
the  gunsight  but  not  the  predictor.  The 
narrative  content  of  the  demonstration  was 
provided  by  an  instructor  pilot  seated  in 
the  T-37  cockpit  beside  the  subject. 
Following  presentation  of  the  recorded 
demonstration,  the  instructor  exited  the 
cockpit  and  all  further  instruction  was 
accomplished  from  the  instructor/operator 
console.  The  only  exception  to  this 
procedure  was  for  the  no  gunsight  group, 
where  in  order  to  familiarize  the  subject 
with  the  use  of  the  gunsight,  a second 
demonstration  (this  time  with  gunsight)  was 
presented  between  trials  5 and  6. 

All  groups  performed  15  trials  without 
interruption.  For  the  Standard  Group,  all 
15  trials  were  performed  with  the  gunsight, 
but  not  with  the  predictor  cue  present. 

For  the  predictor  group,  the  predictor  cue 
was  continuously  present  during  all  15 
trials.  For  the  No  Gunsight  group,  the 
predictor  cue  was  never  available.  For  the 
first  five  trials,  the  gunsight  was  not 
available  either.  For  the  No  Gunsight 
group,  the  gunsight  was  introduced  on  trial 
6 and  was  present  for  all  remaining  trials. 

Instructors  seated  at  the  console  were 
given  a planar  view  of  the  ground  track  and 
final  leg  segment  of  the  maneuver  as  well 
as  a graphic  display  of  the  bomb  circle  and 
impact  point  with  indications  of  the 
following  release  parameters  (airspeed, 
heading,  altitude,  dive  angle,  g-load).  No 
restrictions  or  specific  instructions  were 
given  to  the  instructor  as  to  the  manner  in 
which  this  information  should  be  used. 

Following  the  15th  trial,  subjects 
exited  the  simulator  for  a short  break  and 
final  critique  by  the  instructor  prior  to 
reentering  the  cockpit  for  the  10  final 
trials  which  were  conducted  in  the  absence 
of  any  instructor  feedback.  For  the  final 
10  trials,  subjects  in  all  three  groups 
performed  under  the  same  conditions  (i.e., 
gunsight,  no  predictor  cue,  and  no 
instructor  feedback).  Dependent  measures 
collected  consisted  of  "ircular  error  and 
release  parameters. 
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RESULTS 


The  results  of  the  present  study  will 
be  presented  in  three  sections.  The  first 
section  will  examine  the  relationship  of 
various  measures  of  flying  experience  to 
circular  error  measures  of  bombing 
performance.  The  second  section  will 
present  differences  between  treatment 
groups  in  terms  of  measures  of  circular 
error.  The  third  section  will  present 
differences  between  treatment  groups  in 
terms  of  release  parameters. 

Measures  of  Flying  Experience  and  Bombing 
Performance 

Prior  to  comparing  treatment  groups  in 
terms  of  bombing  performances,  groups  were 
compared  for  possible  significant 
differences  in  flying  experience.  Neither 
of  three  different  measures  of  performance 
were  found  to  be  significantly  different. 
Groups  did  not  differ  in  terms  of  total 
flying  hours  (F(2, 19)=0.5676,  p>  .05);  T-37 
flying  hours  (F(2, 19) =3 . 30332,  p>.05);  or 
the  ratio  of  T-37  to  total  flying  hours 
(F(2, 19)  = 1 . 35116,  p>  .05) . With  one 
exception,  none  of  the  above  measures  of 
flying  experience  was  found  to  be 
correlated  significantly  with  bombing 
performance,  when  bombing  performance  was 
taken  as  the  mean  circular  error  over 
trials  20-25.  The  one  exception  was  for 
the  predictor  group,  where  the  ratio  of 
T-37  to  total  flying  time  was  found  to  be 
significantly  correlated  with  circular 
error  (r=-.9165,  df=4,  p <.05 ) for  trials 
20-25.  Comparisons,  however,  between  this 
measure  of  flying  experience  for  subjects 
in  the  predictor  group  with  performances 
over  Blocks  B1-B3  did  not  reveal  the 
presence  of  a significant  relationship. 
Thus,  despite  the  wide  range  of  flying 
experience  represented  across  the  three 
subject  groups,  the  results  indicate  that 
the  experience  variable  was  not 
systematically  related  to  performances  in 
the  present  study. 

Differences  in  Circular  Error  Scores 

The  results  of  primary  concern  deal 
with  the  differences  between  treatment 
groups  in  terms  of  mean  circular  error.  As 
can  be  seen  in  Figure  1,  all  groups  show  a 
significant  decrease  in  circular  error 
scores  over  the  first  15  trials.  Mean 
circular  error  and  standard  deviation  in 
circular  error  by  blocks  of  trials  are 
given  in  Table  2. 


TABLE  2 

MEAN  CIRCULAR  ERROR  (FEET) 


Group 

b locks 

Five 

rr  aU 

— 

1-5 

6-10 

11-  15 

16-20 

21^25 

Stand 

(N-ll) 

X* 

s.d.« 

314.74 

251.99 

227.14 

177.12 

205.22 

129.22 

144.80 

96.25 

192.76 

111.63 

Pred 

(N=6) 

X- 

s.d.« 

405  13 
247.49 

237.87 

167.42 

212.60 

164.22 

316.67 

172.05 

170.13 

138.63 

No  Gun 
Slqht 

!«-5) 

x» 

s.d.» 

456.76 

282.86 

152.28 

122.67 

126.60 

83.95 

170.40 

89.68 

167.20 

76.85 

Not  only  was  an  improvement  in 
accuracy  noted  over  the  first  three  blocks 
of  trials,  but  also  a decrease  in  the 
variability  of  the  bombing  performances  . 
The  difference,  however,  between  circular 
error  scores  for  the  predictor  and 
standard  groups  was  not  found  to  be 
statistically  significant  (F( 1,15) =0 . 363 , 
p= . 56 19).  Neither  was  the  difference 
between  the  standard  group  and  the 
no-gunsight  group  statistically  significant 
(F(l,14)=. 004 , p= . 9469 ) . The  failure  to 
find  a difference  between  the  standard  and 
no-gunsight  groups  is  probably  accounted 
for  by  the  poor  first  block  performance  of 
the  group  performing  the  task  without  the 
gunsight.  When  a comparison  is  performed 
between  the  standard  and  no-gunsight  groups 
for  blocks  2 and  3 (i.e.,  trials  6-15),  the 
difference  approaches  statistical 
significance  (F(l,14) =2 . 528 , p= . 1274) . 

Discontinuation  of  instructor  feedback 
produced  no  significant  effect  upon 
the  performances  of  the  no-gunsight  and 
standard  groups.  It  must  be  remembered 
that  the  no-gunsight  group  and  the  standard 
group  were  practicing  under  identical 
conditions  for  trials  6-15.  As  is  seen, 
however,  in  Figure  1,  abrupt  removal  of  the 
predictor  cue  produced  an  approximate  50 
percent  decrease  in  accuracy.  This 
decrease  in  accuracy  was  quickly  overcome, 
however,  so  that  by  the  last  block  of  five 
trials,  little  if  any  difference  can  be 
noted  between  the  three  groups.  The 
effect  of  removing  the  predictor  cue  is 
instructive,  inasmuch  as  the  absence  of  any 
difference  between  the  standard  and 
predictor  groups  during  the  first  15  trials 
gives  little  evidence  that  the  cue  was  even 
being  utilized.  The  marked  disruption, 
however,  following  its  removal  gives 
evidence  to  the  contrary. 
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MEAN  CIRCULAR  ERROR  (FEET) 


CONDITIONS 


Figure  1 

Circular  Error  Across  Trials  As  A Function  Of  Instructional  Conditions 
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Differences  Between  Groups  in  Terms  of 
Release  Parameters' 

Means  and  standard  deviations  for  each 
of  the  five  release  parameters  are  given  in 
Table  3. 


Standard  and  predictor  cue  groups  were 
found  to  differ  significantly 
(F( 1,15) =4 .619,  p=.0461 ) in  dive  angle 
variability  over  trials  1-15.  While 
circular  error  scores  were  not  found  to 
differ  significantly  over  these  blocks  of 


TABLE  3 

RELEASE  PARAMETERS 
MEANS 


BLOCKS  OF  FIVE  TRIALS 


RELEASE  PARAMETERS  1-5  6-10  11-15  16-20  21-25 


s 

349.37 

349.84 

349.70 

346.14 

248.90 

HEADING 

P 

347.03 

348.60 

347.23 

348.23 

333.07 

(350  degrees) 

NS 

349.92 

348.92 

349.12 

350.68 

410.00 

S 

3012.95 

2909.58 

2840.92 

2910.65 

2886.32 

ALTITUDE 

P 

2992.37 

2969.99 

2965.83 

2965.07 

2984.70 

( 3000 ' AGL) 

NS 

2965.48 

2839.64 

2932.43 

2965.43 

3012.99 

S 

1.10 

1.18 

1.12 

1.11 

1.20 

G LOAD 

P 

1.32 

1.23 

1.28 

1.28 

0.90 

(0.09) 

NS 

1.08 

1.20 

1.18 

1.30 

1.21 

S 

308.59 

308.71 

309.94 

309.25 

308.64 

AIRSPEED 

P 

308.80 

306.40 

304.87 

307.60 

303.90 

(300  KIAS) 

NS 

309.96 

312.48 

310.24 

311.36 

307.64 

S 

28.00 

29.16 

29.70 

30.11 

29.56 

DIVE  ANGLE 

P 

28.83 

29.00 

27.33 

28.00 

28.03 

(30  degrees) 

NS 

30.12 

29.24 

29.52 

29.12 

27.84 

STANDARD  DEVIATIONS 

BLOCKS  OF  FIVE  TRIALS 

RELEASE  PARAMETERS 

1-5 

6-10 

11-15 

16-20 

21-25 

S 

3.17 

2.78 

2.39 

6.45 

2.45 

HEADING 

P 

2.55 

3.12 

4.66 

2.58 

2.25 

NS 

3.84 

1.91 

1.64 

2.02 

1.54 

S 

311.63 

264.28 

204.79 

149.11 

161.02 

ALTITUDE 

P 

283.30 

392.65 

283.94 

168.88 

186.86 

NS 

206.56 

151.77 

131.28 

131.17 

136.81 

S 

0.50 

0.48 

0.31 

0.16 

0.31 

G LOAD 

P 

0.41 

0.35 

0.42 

0.42 

0.28 

NS 

0.39 

0.33 

0.29 

0.22 

0.33 

S 

10.05 

7.47 

4.77 

6.84 

4.72 

AIRSPEED 

P 

7.76 

8 93 

9.27 

4.48 

5.30 

NS 

6.34 

4.59 

4.76 

4.76 

3.24 

S 

3.27 

2.78 

2.21 

2.50 

2.21 

DIVE  ANGLE 

P 

4.85 

3.23 

3.38 

2.54 

2.11 

NS 

3.11 

2.10 

1.80 

1.39 

1.86 

trials,  the  predictor  group  showed  as  much 
as  53  percent  more  variability  in  dive 
angle  (block  3)  than  the  standard  group. 
Comparisons  between  the  standard  and 
predictor  groups  for  trials  16-25  when  the 
predictor  cue  was  removed  and  all  subjects 
performed  in  the  absence  of  any  instructor 
feedback  showed  a tendency  for  mean  dive 
angle  to  be  shallower  for  the  predictor 
group  than  for  the  standard  group.  This 
difference,  however,  was  not  statistically 
significant  (F( 1 , 15) =2 . 887 , p= . 1 068 ) . In 
attempting  to  isolate  from  measures  of 
release  parameters  the  basis  for  the 
disruption  in  accuracy  caused  by  removal  of 
the  predictor  cue,  groups  were  compared 
over  block  B3  and  block  Tl.  The  only 
difference,  in  terms  of  release  parameters, 
that  was  identified  was  a significantly 
greater  variability  in  g-loading  for  the 
predictor  group  as  compared  to  the  standard 
group  ( F ( 1 , 1 5 ) =4 . 932 , p=.0402).  While 
g-loading  was  approximately  35  percent  more 
variable  for  the  predictor  group  on  block 
B3,  removal  of  the  predictor  cue  on  block 
Tl  caused  the  variability  in  g-load  for  the 
predictor  group  to  increase  to 
approximately  three  times  that  of  the 
standard  group. 

While  differences  between  the  standard 
group  and  the  no  gunsight  group  were  marked 
in  terms  of  circular  error,  differences  in 
cerms  of  release  parameters  were  more 
subtle.  In  fact,  only  one  comparison 
between  the  two  groups  for  trials  1-5 
revealed  a difference  that  even  approached 
statistical  significance.  This  was  in  the 
case  of  altitude  variation  upon  release 
where  there  was  a trend  (F(l,14) =2 . 834 , 
p= . 1 1 64 ) toward  greater  variation  in 
altitude  in  the  standard  group  as  compared 
to  the  no  gunsight  group. 

DISCUSSION 

Comparisons  of  performances  in  the 
present  study  with  those  reported  by  Gray 
and  Fuller  (1977)  reveal  an  approximate  30 
foot  difference  in  mean  circular  error  for 
subjects  performing  the  30  degree  task 
under  similar  conditions.  In  light  of  the 
variability  associated  with  individual 
performances  in  the  present  study,  a 30 
foot  difference  would  be  considered  to  have 
occurred  by  chance.  It  must  be  remembered 
too  that  the  Gray  and  Fuller  (1977)  study 
was  conducted  to  demonstrate  the  limit'  to 
which  this  type  of  task  could  be  taughc  in 
the  simulator  and  in  so  doing  employed 
experienced  instructors  within  the  context 
of  a developed  syllabus.  The  present  study 
employed  the  air-to-surface  task  because  of 


its  convenience  as  a benchmark  task  against 
which  alternative  instructional  treatments 
could  be  evaluated.  Therefore,  the  level 
of  performance  attained  was  secondary  to 
the  sensitivity  of  the  task  to  any  main 
effects  in  terms  of  instructional  treatment 
conditions. 

These  data  are  instructive  too  for 
several  reasons.  First,  from  the 
standpoint  of  flying  traininq  ■ . • ■ 

these  data  demonstrate  that  act’.e  ■ • 
over  cues  inherent  in  the  visua’ 
environment  of  the  student  in 
the  gun  sight  itself)  may  lead  t t"  ' • 
performance  than  the  augmentat  >r  if  *>  •• 
environment  with  cues  intended  t i • 
student  in  performing  a difficult  1 1 ,* 
the  present  study,  control  over  st '■>  , 
the  student's  visual  environment  pr  ... 
be  more  effective  than  attempts  to  ai'n 
the  difficulty  of  a complex  tracking  task 
through  augmented  visual  feedback. 

Secondly,  the  present  data  clearly 
showed  that  the  abrupt  removal  of  an 
augmented  visual  cue  ( i . e . , the  predictor 
cue)  can  produce  a significant  decrease  in 
accuracy  and  an  accompanying  increase  in 
variability  when  no  provision  exists  for 
gradually  fading  out  that  cue.  The  results 
of  the  present  study  clearly  point  out  the 
need  in  future  systems  for  more  active 
control  over  the  stimuli  present  in  the 
training  environment.  . . not  only  their 
presence  or  absence,  but  also  their 
discriminabi lity  (e.g.,  intensity,  etc.). 

It  is  clear  too  that  before  such  active 
control  can  be  incorporated  into  adaptive 
approaches  e.g.,  Williges  and  Williges, 
(1977),  research  must  address  how  such 
changes  are  most  effectively  made 
contingent  upon  student  performance. 

A third  point  concerns  the  poor 
perfomance  obtained  from  subjects  using  the 
predictor  cue.  While  the  disruption  in 
performance  upon  removal  of  the  cue  was 
expected,  the  failure  of  the  predictor 
group  to  initially  outperform  the  standard 
group  was  totally  unexpected.  While 
subjects  were  instructed  to  treat  the  cue 
as  an  "aid"  that  after  15  trials  would  be 
removed,  the  evidence  is  clear  from  the 
disruption  that  occurred  on  Block  Tl  that 
the  cue  was  being  used  during  practice  on 
Blocks  B1-B3.  It,  thus,  does  not  appear  to 
be  the  case  that  the  instructions  biased 
subjects  in  the  direction  of  not  using  the 
cue.  It  may  be  that  although  such  a cue  may 
serve  to  facilitate  the  performance  of  the 
naive  student,  its  use  with  experienced 
pilots  such  as  those  in  the  present  study 
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served  to  increase  rather  than  decrease  the 
difficulty  of  the  air-to-surf ace  task. 

The  results,  however,  do  not  preclude 
the  potentially  effective  application  of 
such  predictor  cues  with  pilots  of  lesser 
experience.  For  example,  a visual  cue 
similar  to  that  used  in  the  present  study 
but  instead  depicting  the  "aimpoint"  of  the 
aircraft  (i.e.,  the  point  at  which  the 
aircraft  would  hit  the  ground  given  its 
present  configuration)  might  prove  to  be 
effective  not  only  in  the  bombing  task  but 
in  acquisition  of  the  landing  task  as 
well.  Research  is  continuing  at  the  Air 
Force  Human  Resources  Laboratory's  Flying 
Training  Division  into  different  and  more 
effective  means  of  manipulating  the  visual 
environment  of  the  pilot. 
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ADVANCED  SIMULATION  FOR  NEW  AIRCRAFT 
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Williams  Air  Force  Base,  Arizona 


ABSTRACT 

The  traditional  procurement  process 
for  new  military  aircraft  simulators 
results  in  a long,  costly,  and  dangerous 
delay  in  availability  of  training 
equipment,  after  introduction  of  the 
aircraft.  The  Advanced  Simulator  for 
Pilot  Training  (ASPT)  of  the  Human 
Resources  Laboratory,  Flying  Training 
Division  has  been  modified  to  provide 
early  simulation  of  the  A-10  and  F-16 
aircraft.  The  resulting  advance  in  A-10 
program  development  has  been  dramatic. 
Although  not  yet  fully  operational,  the 
ASPT  F-16  simulation  will  provide  at  least 
comparable  benefits  for  F-16  training 
program  development.  The  ASPT 
modification  program  demonstrates  a 
reasonable  method  of  greatly  improving 
availability  and  effectiveness  of 
simulator  training  programs. 

THE  PROBLEM 

Military  aircraft  simulators,  more 
aptly  described  as  weapon  system  trainers, 
are  an  integral  part  of  the  modern 
operational  flight  training  system.  These 
devices  provide  the  capability  to  acquire 
(and  practice)  outside  the  aircraft, 
skills,  both  cognitive  and  motor,  critical 
to  the  successful  completion  of  tasks 
ranging  from  aircraft  transition  training 
to  advanced  weapons  delivery  and 
air-to-air  combat;  however,  current 
simulator  procurement  schedules  seriously 
limit  simulator  design  and  effective 
utilization  within  the  overall  training 
program.  At  present,  the  time  delay  from 
the  initial  integration  of  the  aircraft 
into  the  operational  inventory  until  the 
corresponding  simulator  system  is 
available  for  training  is  normally  about 
five  years.  This  lag  adversely  affects 
not  only  the  energy,  equipment,  and 
manpower  costs  within  the  operational 
training  schedule,  but  also  the  design  of 
the  weapon  system  trainer  itself. 

Virtually  every  important  simulator  (and 
syllabus)  design  decision  must  be  based  on 
experience,  judgement,  or  guesswork  - 
everything  except  concrete  evidence 
founded  upon  first-hand  knowledge. 

Although  simulation  is  widely  used  during 
preliminary  engineering  design  and 
aircraft  development,  these  simulations 
are  not  suitable  for  training  system 
development.  These  inputs  are  provided 
during  the  first  few  years  of  operational 


development  of  a given  aircraft,  when 
often  by  trial  and  error,  marked  at  times 
by  loss  of  life,  training  problems  are 
gradually  eliminated.  We  believe  that 
this  dangerous  transition  period  can  be 
eliminated  by  using  a training  system 
development  simulator.  A reasonable 
simulation  of  the  new  aircraft  coincident 
with  operational  deployment  could  be  used 
to  address  training  issues,  as  well  as 
simulator  design  issues  prior  to  their 
requirement  for  use. 

SOLUTION 

The  solution  to  this  problem  involves, 
as  we  see  it,  the  following  organizational 
and  engineering  elements: 

a.  A simulation  facility  (or 
combination  of  facilities)  with  sufficient 
potential  to  construct,  in  a short  period 
of  time,  reasonable  models  of  a wide 
variety  of  aircraft  and  aircraft  task 
environments . 

b.  A close  cooperation  between  the 
aircraft  development  group,  the  Command 
Instructional  System  Development  team,  the 
simulator  manufacturer  having 
responsibility  for  the  final  weapon 


FIGURE  1. 

ADVANCED  SIMULATOR  FOR  PILOT  TRAINING 
(ASPT),  SHOWING  60  INCH,  SIX  DEGREE  OF 
FREEDOM,  SYNERGISTIC  PLATFORM  MOTION 
SYSTEM  AND  WRAPAROUND  COMPUTER  IMAGE 
GENERATED  VISUAL  DISPLAY. 
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systems  trainer,  and  the  designated 
facility  where  the  preliminary  training 
and  research  will  occur. 


The  simulation  facility  should 
support,  as  a minimum,  the  following  basic 
systems; 


(1)  Flight  dynamics. 


VECTOR  GENERAL  GRAPHICS  DISPLAYS,  AND  A 
WIDE  VARIETY  OF  INSTRUMENTS,  RECORDERS  AND 
CONTROLS,  ENTIRE  MISSION  PROFILES  CAN  BE 
CAREFULLY  CONTROLLED. 

The  HRL  ASPT  has  all  of  these  features 
and  is  admirably  suited  for  training 
program  research  and  development,!  Figures 
1-6.) 


(2)  Basic  aircraft  instruments 
and  controls. 

(3)  Wide  field  of  view,  high 
resolution  visual . 

The  systems  ( 1 ) - ( 3 ) are  absolutely 
required  in  all  critical  aircraft  tasks. 
Additionally,  to  be  effective,  a research 
facility  should  have  most  or  all  of  the 
following: 

(4)  Six  degrees  of  freedom 

mot i on . 

(5)  G seat. 

(6)  G suit. 

(7)  Other  Aircraft/Environment 
systems  such  as  sound,  navigation, 
conmunication,  hydraulics,  etc. 

(8)  An  advanced 

instructor-operator  station  with  automated 
student  performance  measures. 


FIGURE  2. 

ASPT  ADVANCED  INSTRUCTOR  OPERATION. 
THROUGH  THIS  STATION,  THE  RESEARCHER 
CONTROLS  THE  ASPT  SIMULATION 
EXPERIMENTATION.  USING  ALPHANUMERIC, 


Most  important  of  all,  the  facility 
should  have  an  extensive  performance 
measurement  capability  backed  by 
sufficient  computer  capacity  to  support 
such  special  research  requirements 
specific  to  a given  operational  aircraft. 


FIGURE  3. 

DISTRIBUTED  PROCESSOR  SIMULATION  SYSTEM. 
THIS  COMPUTATIONAL  NETWORK  PROVIDES  THE 
CAPABILITY  FOR  AN  AIRCRAFT/ENVIRONMENTAL 
SIMULATION  AT  A MINIMUM  OF  30  HZ  FOR  ALL 
AEROSPACE  VEHICLES  EXCEPTING  CERTAIN 
HELICOPTER  SIMULATIONS,  WHILE  PROVIDING 
SUFFICIENT  BACKGROUND  TIME  TO  SUPPORT 
PARALLEL  RESEARCH  AND  ENGINEERING 
DEVELOPMENT  CAPABILITY. 

Care  must  be  taken  to  limit  the  scope 
of  the  simulation  development.  Because 
the  timing  of  the  research  is  critical  to 
the  vitality  of  the  entire  projects,  we 
recommend  a multiphase  effort  based  on  the 
critical  task  approach.  Within  this 
philosophy,  the  Phase  I simulation  would 
be  a "no  frills"  model  capable  of 
performing,  say,  transition  tasks.  Only 
those  instruments,  controls,  and  accessory 
equipment  essential  to  the  accomplishment 
of  the  Phase  I objectives  need  be 
simulated. 


PAYOFFS 

The  payoffs  associated  with  this 
approach,  as  we  see  them,  are  direct 
force  readiness,  as  measured  by  the 
capability  of  a group  of  pilots  to  perform 
their  assigned  mission,  will  be  greatly 
increased.  Transition  training  will  be 
safer  and  more  efficient.  Emergency 
situation  training,  perhaps  impossible  to 
practice  safely  in  the  aircraft,  can  be 
provided  in  the  simulator.  The  greatest 
areas  of  training  payoff  will  naturally  be 
associated  with  the  principally  cognitive 
tasks  (such  as  those  in  air-to-air  or 
air-to-surface  attack).  In  addition  to 
the  enhanced  safety  and  improved  training, 
large  amounts  of  fuel,  manpower,  and 
equipment  dollars  will  be  conserved,  a 
real  bonus  for  the  operational  commander. 
At  the  same  time,  research  payoffs  will  be 
equally  great.  Major  design 
considerations  for  the  weapon  systems 
trainer  can  be  fully  determined  and  this 
information  provided  to  the  simulator 
contractor.  High-cost  hardware  issues, 
such  as  visual  system  design,  instructor 
operator  station  design,  and  even  motion 
cueing  systems  alternatives,  can  be 
effectively  determined.  At  the  same  time, 
issues  related  to  both  simulator  and 
aircraft  utilization  for  operational 
training,  including  syllabus  development, 
course  content,  task  sequencing, 
development  and  validation  of  performance 
assessment  measures  will  be  possible. 

RESULTS 

To  date,  the  Flying  Training  Division, 
Air  Force  Human  Resources  Laboratory,  has 
successfully  applied  the  approach  outlined 
in  this  paper  to  the  A-10  aircraft,  using 
the  ASPT  as  the  development  simulator,  and 
is  preparing  a similar  effort  for  the  F-16 
aircraft.  The  A-10  Phase  I simulation  on 
the  ASPT,  while  austere,  has  been 
successful.  In  general  terms,  the 
objectives  of  the  ASPT  A-10  Phase  I 
conversion  were: 

(1)  To  provide  interim 
transition  training  for  A-10  pilots  in  the 
period  before  delivery  of  the  A-10  WST. 

(2)  To  provide  introductory 
surface  attack  training  during  the  same 
per i od . 

(3)  To  assess  advanced  training 
features  of  the  ASPT  instructor-operator 
console  for  possible  adaptation  to  future 
fighter/attack  simulators. 

(4)  To  develop  automated 


objective  performance  measures  of 
f ighter/attack  pilot  performance. 

(5)  To  develop  the  A-10 
simulator  training  program  and  provide 
hands  on  experience  for  A-10  simulator 
instructor  pilots. 


FIGURE  4. 

ASPT/A-10  PHASE  I COCKPIT  SIMULATION.  THIS 
PICTURE  DEMONSTRATES  THE  LEVEL  OF 
SIMULATION  REQUIRED  TO  ACHIEVE  TRANSFER  OF 
TRAINING  GOALS  FOR  PHASE  I.  PICTURED 
ELEMENTS  INCLUDE  PRINCIPAL  AIRCRAFT 
CONTROLS,  INSTRUMENTS,  AND  HEADS  UP 
DISPLAY. 

All  of  these  objectives  were  met 
satisfactorily;  in  most  cases,  results 
were  considerably  better  than  were 
expected  from  the  rather  austere  Phase  I 
simulation. 

TRANSITION  TRAINING 

By  May  1978,  17  B course  students  had 
received  transition  training  in  the  ASPT 
A-10.  All  of  these  pilots  successfully 
transitioned  into  the  airplane.  Although 
numbers  are  too  small  for  statistical 
significance  at  this  time,  it  appears  that 
the  ASPT  trained  pilots  are  the  equivalent 
of  about  two  aircraft  sorties  ahead  of 
where  they  would  be  without  ASPT 
training.  This  undoubtedly  is  at  least 
partially  due  to  the  extensive  cooperative 
effort  by  instructor  pilots  from 
Davis-Monthan  AFB  and  AFHRL  engineers  to 
assure  that  the  A-10  performance  and 
handling  qualities  were  as  faithfully 
simulated  as  available  data  would  permit. 


The  transition  training  results  for 
the  first  class  were: 

(1)  Of  47  rides,  only  one  failed. 

(2)  In  the  first  67  rides,  there 
were  no  unsatisfactory  patterns  or 
landings. 

(3)  In  the  first  67  rides: 

5 OX  rated  01  (Fully 

Qualified) . 

50X  rated  02  (Qualified  with 
Additional  Training). 


FIGURE  5. 

ASPT  PHASE  I RUNWAY  SIMULATION.  THIS 
AUSTERE  DATA  BASE  SHOWS  THE  LEVEL  OF 
DETAIL  REQUIRED  FOR  SUCCESSFUL  TRANSITION 
TRAINING  FOR  THE  A-10  PROGRAM. 

INTRODUCTORY  SURFACE  ATTACK  TRAINING 

Results  of  the  introductory  surface 
attack  training  are  spectacular.  The  ASPT 
A-10  proved  to  be  exceptionally  effective 
for  bomb  and  gunnery  range  instruction  - 
in  fact,  the  simulator  is  more  effective, 
sortie  for  sortie,  than  the  airplane. 

The  surface  attack  training  results 
for  the  first  class  are: 


30  Deg  Dive  Bomb 
20  Deg 


Needed  to 
Qualify  (C 


First  Class 
Average  (CEP) 


A number  of  factors  probably 
contribute  to  the  high  effectiveness  of 
the  ASPT  A-10.  Prerecorded  demonstration 
runs  show  the  student  exactly  how  to 
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FIGURE  6. 

CONVENTIONAL  BOMBING  RANGE.  THIS  PICTURE 
SHOWS  THE  APPROXIMATE  LEVEL  OF  DETAIL 
REQUIRED  TO  TRAIN  CONVENTIONAL  BOMBING 
TASKS. 

achieve  best  results.  Using  freeze  and 
reset  features,  many  more  practice  runs 
are  possible  in  the  simulator  than  in  the 
airplane.  The  student  is  immediately 
informed  of  the  results  of  each  run,  as 
well  as  the  exact  conditions  that  existed 
at  release  time. 

ADVANCED  TRAINING  FEATURES 

As  mentioned  above,  the  ability  to 
record  and  play  back  demonstration  flights 
appears  to  have  great  training  value, 
particularly  for  single  seat  aircraft  such 
as  the  A-10.  The  related  capability  to 
record  a student  flight  and  play  it  back 
later  for  a critique  also  seems  to  have 
great  promise. 

The  ability  to  freeze  the  action  at 
any  point  and  to  reset  rapidly  to  chosen 
initial  conditions  allows  concentrated 
practice  and  assessment  of  procedural 
errors . 

Use  of  automated  performance  measures 
(discussed  below)  assures  objective 
evaluation  of  performance. 

A number  of  such  training  features 
were  evaluated  during  the  ASPT  A-10  Phase 
I,  and  recommendations  for  future 
fighter/attack  simulators  are  being 
developed.  This  research  will  continue 
during  Phase  II.  Evaluation  of  the 
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training  value  of  such  features  as  “faster 
than  real-time"  operation  and  special 
training  displays  such  as  continuous 
display  of  impact  point  is  under 
consideration. 

AUTOMATED  PERFORMANCE  MEASURES 

There  are  many  reasons  for  developing 
automated  measures  of  student  performance: 

(1)  The  student  is  assured  of 
accurate,  impersonal,  objective  assessment 
of  his  performance. 

(2)  Many  more  aspects  of 
performance  can  be  evaluated  automatically 
than  is  possible  for  a single  instructor 
pilot. 

(3)  Instructor  pilot  workload  is 
eased,  permitting  him  to  devote  his 
attention  to  instructional  duties. 

In  general,  performance  measures  must 
be  specially  developed  for  each  aircraft 
and  each  task.  Following  is  a list  of 
A-10  tasks  for  which  automated  performance 
measures  have  been  developed  and  used: 

A-10  PERFORMANCE  MEASUREMENT  TASKS 
TRANSITION 

Takeoff,  climb  and  level  off 
Slow  Flight 

Lazy  8,  Aileron  Roll  (clean  & 40% 
speedbrake) 

Loop,  Cuban  8,  Split  S 

Simulated  flameout  pattern 

Straight-in  pattern 

Normal  overhead  (360°)  pattern 

Closed  normal  pattern 

Closed  no  flap  pattern 

Closed  simulated  single  engine  pattern 

Re-entry  to  normal  overhead  pattern 

AIR-TO-GROUND 
30  Degree  Dive  Bomb 
20  Degree  Low- Angle  Low- Drag  Bomb 


15  Degree  Low-Angle 
Low-Angle  Strafe 
High-Angle  Strafe 
Low-Angle  Low-Drag  Pop-Up 
Low-Angle  Pop-Up 
Low-Angle  Strafe  Pop-Up 
Hung  Bomb  Pattern 

SIMULATOR  TRAINING  PROGRAM 

At  this  time,  the  A-10  simulator 
training  syllabus  has  been  used  with  17 
students.  Ten  instructor  pilots  have 
participated  in  the  training  program. 
Without  the  ASPT  A-10,  this  status  would 
not  have  been  achieved  for  at  least  two 
more  years.  Experience  with  an  A-10 
simulator  including  a full  visual  system 
would  not  be  possible  for  at  least  five 
years.  Thus  the  A-10  training  program  has 
been  advanced  by  2-5  years,  with  a very 
substantial  improvement  of  the 
effectiveness  of  the  A-10  aircraft. 

CONCLUSIONS 

We  have  presented,  and  to  a large 
degree,  tested  and  verified  a managerial 
tool  for  solving  the  problem  of  time  delay 
between  the  deployment  of  an  operational 
aircraft  and  its  corresponding  weapon 
system  trainer.  This  approach  is  seen  to 
have  large  payoffs  in  terms  of  direct 
research  and  design  benefits,  as  well  as  a 
positive  impact  force  readiness,  pilot 
safety,  and  fuel  expenditure.  We  have 
restrained  throughout  this  discussion  from 
detailing  any  particular  potential 
engineering  mechanization  of  this  plan,  as 
such  a mechanization  depends  primarily  on 
the  aircraft  (or  system)  simulated  and  the 
engineering  characteristics  and  capability 
of  the  host  facility.  We  feel  that  the 
procedure  presented  is  a natural  extension 
of  the  use  of  simulation  to  solve  training 
problems  and,  as  such,  should  become  a 
permanent  part  of  all  future  aircraft 
development  and  deployment  schedules. 
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MICROCOMPUTER  BASE  FOR  CONTROL  LOADING 

DR.  F.  GERRY  ALBERS 

School  of  Engineering 
University  of  Dayton 


INTRODUCTION 

Simulator  designers  have  been  faced 
with  two  familiar  problems  throughout  the 
digital  age  of  simulation;  namely,  framing 
time  crunch  and  discrete  system  anamolies 
associated  with  models  of  analog  systems. 
This  project  has  been  oriented  to  impact  both 
of  these  problem  areas  as  well  as  to  produce 
a piece  of  gear  suitable  for  general  simula- 
tor usage.  The  primary  rationale  for  con- 
ducting this  research  has  been  to  distribute 
the  intelligence  of  the  simulator  to  points 
where  it  is  needed  and  thus  relegate  the  host 
computer  to  the  role  of  a system  manager. 
The  control  loading  task  was  selected 
because  of  its  suitability  for  distributed  pro- 
cessing and  because  of  its  need  for  frame 
rates  higher  than  the  nominal  15  to  30 
frames- per- second  simulator  rate.  The 
U.  S.  Air  Force^specified  that  the  results  of 
this  effort  must  contain  data  from  which  it 
can  write  specifications  and  select  future 
simulator  configurations. 

THE  CONTROL  LOADING  TASK 

Pilots  have  always  and,  no  doubt,  will 
always  complain  that  the  simulator  doesn't 
quite  "feel"  like  the  aircraft.  This  tendency 
for  precise  control  on  the  part  of  the  pilots 
is  often  offset  by  the  astute  engineer  who 
taps  on  the  side  of  a control  cabinet  and  then 
says  "How's  that  feel  now?  " 

The  "feel"  which  the  pilot  experiences 
in  his  control  system  is  usually  composed  of 
six  independent  forces:  spring,  breakout, 
travel  limit,  viscous  friction  (damping). 
Coulomb  friction,  and  velocity  limit.  The 
first  three  forces  are  functions  of  displace- 
ment whereas  the  later  three  are  velocity 
functions.  Occasionally  inertia,  which  is  an 
acceleration  term,  is  also  required  for 
complete  control  synthesis.  Another  indis- 
pensible  parameter  is  the  deadband  which 
the  pilot  experiences  around  a variable  trim 


setting.  In  addition,  the  airframe  may  very 
well  influence  this  "feel"  through  actuator, 
bobweight  and/or  control  surface  force  feed- 
back. 

The  selected  hardware  configuration 
for  this  project  includes  a force-type  control 
loader  equipped  with  position  and  velocity 
transducers  as  shown  in  Figure  1.  A micro- 
computer is  employed  to  calculate  the 
various  forces  mentioned  above  based  on 
inputs  from  the  control  loader's  transducers 
and  selected  parameters  from  the  "host" 
computer,  a Honeywell  316.  The  control 
loader  selected  for  the  task  was  the 
McFadden  Electronics  392A^  3-axis  control 
loader  and  the  microcomputer  was  an  Intel 
System  80/20.  In  order  to  provide  the 
project  with  direction  and  credibility,  the 
A- 10  was  selected  as  the  study  aircraft. 

This  selection  was  made  primarily  on  the 
basis  of  the  availability  of  suitable  control 
loading  data. 


Figure  1.  General  Configuration  Diagram 

PARAMETRIC  ANALYSIS 

Each  of  the  individual  forces  in  some 
way  models  an  element  of  the  aircraft 
control  system.  These  forces  are,  in 
general,  additive  as  shown  in  Figure  2,  even 
though  they  may  be  the  result  of  nonlinear 
processes.  However,  the  total  solution 
requires  that  these  forces  be  logically  con- 
nected to  accurately  represent  the  fully 
integrated  control  system. 


1 Aeronautical  Systems  Division,  Wright- Patterson  Air  Force  Base,  Ohio. 

2 Control  Loader  was  loaned  at  no-cost  to  the  University  of  Dayton  or  the  U.  S.  Government. 
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follows  the  format  of  Equation  1.  The  com- 
plete pitch  force  equation  then  becomes 

* TOT  ’ ' (Vo*25  ,l8"  <XD*  * 1,5  *‘*n  <*>♦*<  *1  U,DB* 

• prtng  breakout  coulomb  viscous  desdbsnd 


Figure  2.  Force  Component  Diagram  - 
Single  Axis 

For  example,  consider  the  forces 
generated  for  a single  axis  (e.  g.  pitch  con- 
trol) which  has  a deadband  due  to  rigging 
slack  at  the  control  stick.  The  pilot  would 
therefore  experience  nearly  zero  force 
within  the  deadband  which  is  described  as 
follows : 


ftot  = [fsp  + fvi  + fvo  + fbr]U(DB) 


+ ftr  + fvel  + fac 


where  the  above  terms  stand  for  total, 
spring,  viscous,  coulomb,  breakout,  travel 
limit,  velocity  limit  and  aircraft  related 
forces,  respectively . The  unit  operator  "U" 
indicates  that  the  quantities  within  the 
brackets  are  nil  in  the  deadband. 

As  an  alternate  example,  consider  a 
case  where  the  rigging  slack  is  remote  from 
the  stick.  The  Coulomb  friction  due  to  pulley 
drag  may  then  be  placed  outside  the  influence 
of  the  deadband  with  the  resultant  force 
equation 


ftot  = [fsp  + fvi  + Wu(DB) 


+ F +F  +F  +F 
CO  TR  VEL  AC 


These  two  examples  represent  wiring  dif- 
ferences in  an  analog  system  but  only  soft- 
ware differences  in  a digital  system,  an 
important  consideration  in  a research  or 
development  environment.  For  the  A-10 
model,  the  deadband  is  very  small  but 


KtXt  - (3  N^.OZtql 


where  Xp  is  the  displacement  (in  inches) 
past  a travel  limit,  Xpj  is  the  displacement 
past  a deadband  limit  and  X is  the  velocity 
in  inches- per- second.  and  q are  the 

normal  and  angular  accelerations  from  the 
airframe  computer  which  produce  the 
control  system  bobweight  effects.  Kg  repre- 
sents five  nonlinear  spring  coefficients 
(breakpoints)  which  characterize  the  A-10. 
Similarly,  Kp  represents  a large  gain 
coefficient  which  produces  the  large  feed- 
back forces  for  small  amounts  of  travel 
(Xp>)  past  the  limit.  The  force  is  calculated 
in  pounds  and,  of  course,  requires  a sign 
change  to  produce  the  required  opposing 
force.  Equation  3 is  obviously  stylized  and 
requires  special  handling  due  to  the  non- 
linearities  involved  and  must  be  properly 
scaled.  The  nonlinearities  are  discussed 
below  and  the  microcomputer  processing, 
which  was  performed  entirely  in  integer 
format,  is  discussed  in  the  software  section. 

In  order  to  handle  the  nonlinearities 
of  such  a system,  the  frequency  response  of 
the  control  loader  must  be  considered.  The 
nominal  30  frames-per-  second  for  digital 
simulators  is  usually  s ifficient  to  make  the 
pilot  believe  that  he  is  flying  in  a parallel, 
analog  world  as  evidenced  by  his  visual 
displays.  However,  the  pilot's  tactile 
mechanism  is  capable  of  much  higher  fre- 
quency response.  Empirical  studies  con- 
ducted on  the  McFadden  control  loader 
revealed  frequency  components  of  1000  Hz 
and  higher.  As  one  might  expect,  these 
components  are  experienced  at  the  breakout 
and  travel  limits  where  forces  suddenly 
change.  To  satisfy  the  ground  rules  of 
information  theory  put  forth  by  Shannon,  the 
microcomputer  should  ideally  be  framing  at 
a 5000  Hz  rate  or  better.  However,  this 
project  demonstrated  that  quite  acceptable 
results  can  be  had  at  120  frames-per- second 
by  judiciously  choosing  compensation 
schemes.  That  is,  by  optimizing  lead,  lag, 
and  gain  coefficients  in  the  individual  com- 
ponent force  calculations  relatively  sharp 
breakouts  and  stops  were  obtained. 
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Various  techniques  were  attempted  to 
achieve  the  proper  compensation,  including  a 
Tustin  recursion  method  to  approximate  the 
desired  transfer  function.  However,  the 
final  product  was  a result  of  an  educated  cut- 
and-try  effort.  For  example,  consider  the 
travel  limit  force  component  which  can  be 
expressed  mathematically  as  a recursion 
relationship 
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ftr  = kt[xt(n)  + xt(n-  1)]  + klx  (4) 


Figure  3.  Hardware  Configuration 


In  the  digital  implementation,  the  needed  lag 
term  is  obtained  by  employing  both  the  pre- 
sent frame  value  for  travel  limit  displace- 
ment X'p(N)  and  the  previous  frame  value 
X'p(N  - 1).  The  gain  is  controlled  by  the 
value  of  K-p  and  the  needed  lead  term  is 
obtained  from  the  available  velocity  signal  X 
which  is  modified  by  the  constant  K^.  The 
unsealed  magnitudes  of  K-p  and  and  the 
number  of  terms  in  the  recursion  portion  of 
the  lag  component  were  determined  empiri- 
cally with  the  final  values  set  at  K-p  = 6 and 
= 2.  The  breakout  term  of  Equation  3 
required  a slight  lag  compensation  augmented 
by  a lead  term  (Kg  = 0.  3)  and  was  empiri- 
cally structured  as 

fbr  = 2Xd+kb*  £or  2Xd<2-5  <5a> 

= 2.5  sign  (Xd)+KbX  for  2XD>  2.  5 (5b) 

Similarily,  the  Coulomb  friction  term 
required  compensation  to  offset  a tendency 
to  "dither"  because  of  its  dependency  on  the 
sign  of  the  velocity.  In  this  case  a simple 
lag  was  implemented  by  employing  the 
velocity  value  as  follows: 

Fcq  = X for  lx  I <1.5  (6a) 

= 1.5  sign  (X)  for  lxl»1.5  (6b) 

THE  HARDWARE 

One  of  the  important  driving  factors  in 
this  study  was  that  the  hardware  must  be 
composed  of  off-the-shelf  components  to  the 
greatest  extent  possible.  A preliminary 
analysis  showed  that  four  primary  functions 
must  be  supplied  by  the  hardware;  namely, 
central  processing,  analog  input,  analog 
output,  and  high-speed  mathematics. 

Figure  3 illustrates  the  hardware  configura- 
tion which  employs  the  System  80/20  com- 
ponents from  Intel  Corporation.  Each  of 


the  four  blocks  represents  a bus  compatible 
circuit  board  which  resides  in  the  System 
80/20  chassis.  The  only  hardware  modifi- 
cations performed  were  wiring  options  which 
are  incorporated  on  the  individual  boards 
for  customizing  purposes. 

The  CPU  is  an  3-bit  processor  with 
2K  of  RAM  (random  access  memory)  and 
with  8K  of  ROM  (read  only  memory)  and  has 
a clock  cycle  time  of  490  nanoseconds.  The 
analog  boards  convert  analog  signals  with 
12-bit  (0.  025%)  accuracy  and  were  con- 
figured to  handle  conventional  analog  signals 
(^10v)  and  offset  binary  (2’s  complement) 
digital  values.  The  high-speed  mathematics 
unit  provides  both  integer,  and  floating  point 
operations  with  appropriate  16  and  32- bit 
accuracies.  Additionally,  the  CPU  provides 
a teletype  (TTY)  interface  for  changing 
equation  coefficients  and  digital  parts  for 
trim  switch  inputs  and  for  frame-time 
monitor  outputs. 

The  interfaces  with  the  control  loader 
and  host  computer  (as  shown  in  Figure  1) 
were  entirely  analog  due  to  contractual 
requirements  for  generality.  An  ideally 
flexible  system  would  have  a digital  host/ 
microcomputer  interface  for  parameter 
passing  and  software  downloading.  Each  of 
the  three  axes  required  four  input  para- 
meters to  satisfy  its  force  equation  (e.  g. 
Equation  3),  stick  displacement  and  velocity, 
airframe  normal  and  rotational  accelerations. 
The  analog  input  board,  which  has  a capacity 
for  16  analog  inputs,  converted  the  required 
12  analog  values  in  approximately  800  micro- 
seconds. Three  of  the  four  available  analog 
outputs  were  employed  to  drive  the  respec- 
tive control  loader  force  inputs  for  each 
axis,  pitch,  roll,  and  yaw. 
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THE  SOFTWARE 

The  entire  control  loader  software  was 
written  in  the  Intel  high-order  microcomputer 
language  called  PLM,  with  the  exception  of 
the  analog-to- digital  conversion  routine  which 
was  performed  in  assembly  language.  The 
programs  were  structured  into  a utility 
module  and  a simulator  module.  As  the 
name  implies  the  utility  module  provided 
system  support  in  the  form  of  calibration  and 
test  routines,  system  interface  procedures, 
and  system  initialization.  The  simulator 
module  contains  the  frame  rate  generator, 
simulator  initialization  routines,  the  analog 
acan/convert  routine,  and  the  simulator 
equations  themselves.  The  executable 
machine  code  was  placed  in  PROM  (pro- 
grammable read  only  memory). 

Perhaps  the  most  unique  feature  of  the 
software  structure  is  that  the  calculations 
were  carried  out  entirely  in  integer  format 
resulting  in  a very  significant  reduction  in 
processing  time  required  for  each  frame. 

As  seen  in  the  above  equations,  the  only 
mathematical  processes  required  are 
addition,  subtraction  and  multiplication. 

Since  both  the  analog  input  and  analog  output 
boards  operate  with  12-bit  precision,  the 
microcomputer  was  designed  to  employ  16- 
bit  precision  (two  8-bit  words).  The  PLM 
language  and  the  high-speed  math  board  both 
support  16-bit  operations  of  this  nature.  All 
parameters  were  scaled  to  keep  the  values 
within  these  limits.  The  equation  coefficients 
were  scaled  to  be  either  integer  values  or 
fractional  values  less  than  1.  The  fractional 
coefficients  were  rescaled  upward  by  multi- 
plying them  by  2^  (65536).  The  result  of  an 
integer  multiplication  (a  32-bit  value  in  the 
high-speed  math  board)  was  scaled  back  down 
by  655  36  by  taking  only  the  most  significant 
16  bits  from  the  result  register.  The  only 
drawback  to  this  method  of  processing  was  the  . 
unsigned  nature  of  the  multiplication,  thus 
requiring  procedures  for  sign  conversion  oi 
negative  quantities. 

CONCLUSIONS 

A purely  objective  evaluation  of  an 


engineering  system  usually  involves  graphs, 
strip  charts  and  computer  printouts.  Thus, 
the  usual  static  and  dynamic  tests  were 
carried  out  with  excellent  success.  Howeve? 
the  last  word  on  a system  involving  "feel" 
must  be  subjective.  Those  who  were  able  to 
experience  the  "feel"  of  the  control  loader 
employing  the  microcomputer  base  con- 
sidered it  to  be  of  "good"  quality  compared 
to  the  same  system  employing  the  analog 
base  supplied  with  the  McFadden  system 
(which  is  easily  judged  as  excellent).  The 
microcomputer,  of  course,  was  performing 
additional  airframe  related  calculations 
which  the  standard  analog  system  does  not. 
The  only  difference  in  quality  seemed  to  be 
at  the  stops  and  breakouts.  Due  to  the 
limited  frequency  response  of  the  micro- 
computer system,  the  respective  gains 
were  held  relatively  low  for  stability 
reasons.  This  practice  resulted  in  the  dis- 
continuities being  less  "sharp"  than  the 
analog  based  version.  On  the  other  hand, 
the  system  was  judged  by  most  to  be  suitable 
for  direct  application  in  existing  simulators. 

RECOMMENDATIONS 

This  study  has  clearly  demonstrated 
the  feasibility  of  microcomputer  control. 

The  only  consequentional  recommendation 
as  a result  of  this  study  would  be  to  increase 
processing  frame  rate  by  a factor  of  five  to 
six  times.  This  improvement  is  within  the 
capabilities  of  present  technology  but  not 
exactly  "off-the-shelf-"  The  most  imme- 
diate thought  would  be  to  use  a 16-bit 
processor  instead  of  the  8-bit  processor 
but  this  would  result  in  only  a 50%  to  60% 
speed  improvement  without  any  other 
changes.  The  most  significant  improvement 
would  be  to  use  a multiprocessor  architec- 
ture with  a processor  devoted  to  each  axis. 
Another  dramatic  speed  improvement  could 
be  obtained  if  a hardware  multiply  unit  was 
employed  which  performed  signed  (as 
opposed  to  the  present  unsigned)  integer 
multiplication.  If  these  three  changes  were 
coupled  with  improvements  in  micro- 
computer technology  (a  daily  occurrence), 
frame  rates  up  to  1000  per  second  are 
within  reach. 
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SUMMARY 


This  paper  discusses  the  ever-increasing  role 
of  EW  training  as  an  outgrowth  of  the 
expanding  emphasis  on  EW  in  our  national 
defense  program.  It  discusses  the  basic 
requirements  of  EW  aircrew  trainers  and  the 
solutions  available.  One  solution  is  the 
hardware  stimulation  approach,  employing 
racks  of  pulse  and  scan  generators  to  provide 
RF  or  video  inputs  into  operational  EW 
equipment.  The  other  solution  is  the 
software  simulation  approach,  modeling  the 
emitter  environment  and  EW  equipment  in  a 
general-purpose  computer  and  providing 
outputs  to  drive  the  EW  equipment  displays 
and  control  panels.  The  software  approach 
has  significant  advantages  in  hardware  cost 
and  complexity,  instructor/operator  inter- 
face, reliability,  maintainability,  and  ease 
of  update  and  modification.  However,  fears 
of  modeling  the  highly  sophisticated  EW 
equipment  and  environments  have  kept  this 
approach  from  being  used.  This  paper 
demonstrates  that  the  software  approach  can 
be  successfully  employed.  A discussion  of  an 
RWR  training  device  recently  developed  by  ATI 
and  using  the  software  approach  is  contained 
in  this  paper.  Finally,  a discussion  is 
presented  on  the  application  of  the  software 
approach  to  new  and  future  EW  systems. 


INTRODUCTION 


Ever  since  the  development  of  radar,  the  role 
of  electronic  warfare  (EW)  in  our  national 
defense  picture  has  been  growing  at  an 
unprecedented  rate.  New  airborne  and 
ground-based  EW  systems  have  evolved  with 
ever-increasing  capabilities  and  complex- 
ities. Our  pilots  and  EW  system  operators 
have  a tremendous  burden  in  keeping  pace  with 
this  growing  technology.  A major  portion  of 
our  defense  relies  on  their  ability  to 
operate  their  equipment  and  perform  their 
missions  in  varieties  of  wartime  and 
peacetime  conditions.  Therefore,  it  is 
imperative  that  these  EW  system  operators  and 
pilots  obtain  rigorous  and  up-to-date 
training  to  achieve  and  maintain  proficiency 
and  combat  readiness. 


There  are  two  basic  methods  by  which  this 
much  needed  training  can  be  provided.  First, 
it  can  be  accomplished  in  the  field  with 
actual  sites  and  live  emitters.  In  the  case 
of  airborne  EW  training,  this  would  involve 
actual  equipment  operating  against  live 
ground  sites,  as  on  a flight  test  range. 


This  approach  to  EW  training  has  the 
advantage  that  everything  is  real.  The 
tracking,  propagation,  polarization  and 
antenna  installation  effects  are  implicit. 
On  the  negative  side  are  a host  of  items, 
headed  by  cost.  Training  of  a single  person 
requires  the  support  of  an  entire  flight  test 
range  with  operators  at  each  site, 
coordination  personnel  and  large  amounts  of 
fuel.  Furthermore,  the  entire  training 
session  has  limited  realism  in  terms  of  the 
make-up  of  the  scenario,  since  there  is 
currently  no  test  range  that  approaches 
postulated  signal  densities  of  even  the  most 
optimistic  European  or  Middle  East  EW 
engagement  scenarios.  The  lack  of 
portability  of  the  sites  introduces  further 
problems  in  the  ability  to  effectively  train, 
since  the  student  regularly  encounters  the 
same  situation. 


The  same  arguments,  of  course,  apply  to  EW 
training  in  environments  other  than  airborne. 
In  a field  exercise,  mobile  ground-based 
ELINT  sites  can  be  positioned  around  a number 
of  live  sites  to  train  operators  to 
efficiently  intercept  signals  of  interest  and 
determine  signal  characteristics  and  perform 
DF  computations.  This  type  of  training 
suffers  limitations  similar  to  the  airborne 
case.  Coordination  of  personnel  and 
equipment  is  massive  and  results  in  high 
costs.  Furthermore,  its  limited  flexibility 
to  create  and  define  totally  new  and 
different  scenarios  for  the  students  results 
in  reduced  situations  and  intercepts. 


The  alternative  to  training  in  the  field  is 
with  a simulator  using  an  artificial 
environment.  In  the  case  of  airborne  EW 
training,  this  involves  the  incorporation  of 
EW  into  the  flight  simulators,  providing  the 
pilot  with  simulated  EW  environments  and  the 
resulting  display  indications.  The  use  of 
flight  simulators  has  been  accepted  for  many 
years  as  a cost-effective  tool  for  pilot 
training.  Of  equal  or  even  more  significance 
is  that  it  presents  the  only  method  of 
introducing  (under  controlled  conditions) 
malfunctions  or  emergency  conditions,  thus 
training  the  pilot  to  cope  with  rare  and 
potentially  hazardous  circumstances.  EW 
training  presents  a number  of  parallels  to 
flight  simulation  in  terms  of  cost, 
coordination,  and  the  need  to  provide 
training  against  postulated  conditions  which 
cannot  be  generated  in  real-life  without 
potentially  catastrophic  consequences. 
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The  simulation  approach  allows  an  instructor 
to  directly  monitor  and  control  the  EW 
training  exercises.  It  permits  freezing  the 
situation  and  can  provide  an  "instant  replay" 
capability  for  highlighting  and  explaining 
mistakes.  The  instructor  can  set  up 
varieties  of  scenarios  prior  to  training 
sessions  and  can  dynamically  control  and 
modify  the  scenarios  during  the  training 
sessions.  Controlling  the  entire  EW 
environment  and  coordinating  the  EW  training 
with  mission  flight  training  is  a significant 
advantage  to  the  simulation  approach. 

One  of  the  more  interesting  aspects  of 
airborne  EW  training  that  can  be  incorporated 
only  when  using  the  simulation  approach 
involves  the  simulation  of  active  electronic 
countermeasure  (ECM)  equipments  and  their 
effect  upon  hostile  weapons  systems.  These 
ECM  equipments  include  both  noise  and 
deception  jammers,  and  various  expendables 
such  as  chaff  and  flares.  In  field  training 
it  is  difficult,  if  not  impossible,  to  employ 
these  ECM  techniques  and  determine  their 
effectiveness  against  anti-aircraft  artillery 
or  surface-to-air  missile  (SAM)  systems.  A 
trainer  permits  these  situations  to  be 
simulated  and  the  effectiveness  of  jamming 
and  maneuvers  to  be  evaluated.  The  degree  to 
which  this  simulation  is  performed  can  vary 
greatly  from  a rather  superficial  simulation 
to  an  extremely  detailed  model.  A 
superficial  treatment  would  determine  only  if 
the  proper  ECM  were  selected  for  the 
particular  threat  type  and,  if  so,  score  a 
miss.  A more  detailed  simulation  would 
include  SAM  missile  dynamics  and  the  entire 
guidance  command  loop,  including  the  guidance 
computer.  The  degree  of  simulation  employed 
in  any  particular  simulator  depends  upon  the 
overall  system  requirements.  The  important 
point  to  note  is  the  potential  which  is 
possible  with  an  EW  simulator. 

With  the  need  for  EW  simulation  clearly 
established,  planners  are  making  EW 
simulation  an  important  requirement  for 
next-generation  flight  simulators.  This 
paper  is  not  intended  to  extoll  the  virtues 
of  EW  simulation  but  rather  to  explain  how 
this  EW  aircrew  training  can  be  provided  in  a 
cost-effective  manner  with  present 
technology. 

GENERAL  REQUIREMENTS  FOR  AN  EW  AIRCREW 
TRAINER 

To  be  an  effective  training  tool,  there  are  a 
number  of  features  which  must  be  incorporated 
into  an  EW  simulator.  First,  an  accurate 
modeling  of  the  EW  environments  postulated 
for  future  conflicts  is  needed.  Not  only 
must  the  emitter  signal  models  possess 
sufficient  flexibility  needed  to  simulate 
complex  signal  types,  but  the  number  of 
signals  and  their  tactical  usage  must  be 


representative  of  that  expected.  Next,  the 
onboard  EW  equipment  simulation  must  be 
realistic.  In  those  cases  where  the  EW 
equipment  contains  analog  displays,  the 
various  anomalies  such  as  noise,  flicker,  and 
fanning  must  be  accurately  preserved.  In  the 
case  of  computer-based  systems  which 
typically  contain  alphanumeric  displays,  the 
display  itself  can  be  duplicated;  however, 
any  failure  modes  of  the  system  such  as 
overloading,  misidentification  and  response 
times  must  be  accurately  duplicated.  All 
major  audio  and  visual  cues  derived  from 
normal  operation  and  failure  operation  of  the 
equipment,  including  their  anomalies,  must  be 
included  in  the  simulation  for  adequate 
trai ning. 

Proper  generation  of  the  display  and 
operation  of  the  controls  is  only  part  of  the 
EW  simulation  and  training  problem.  Another 
part  involves  the  instructor  and  his  ability 
to  set  up  situations  and  monitor  the 
student's  responses.  The  instructor  must 
first  have  some  convenient  and  rapid  method 
of  inserting  and  deleting,  as  well  as 
monitoring  and  controlling  the  modes  of 
operation  of  emitters.  Some  of  these 
emitters  may  be  attack  aircraft,  and  some 
method  of  specifying  their  initial  location 
and  engagement  profile  is  required.  In  the 
ground  case,  emitters  may  also  be  mobile; 
again,  it  is  necessary  to  specify  the 
position  and  expected  movements.  The  modes 
of  emitter  operation  such  as  searching  and 
tracking  and  the  activation  of  missile 
guidance  and  launches  must  also  be 
controllable  by  the  instructor.  Emitter 
modes  could  be  automated  (canned)  to  reduce 
the  instructor's  burden  during  a tactical 
training  mission,  but  should  also  be  manually 
controllable  for  specific  training  exercises. 

Another  requirement  of  an  EW  aircrew  trainer 
is  a real-time  Interface  to  flight  training 
devices.  Learning  how  to  operate  the 
equipment  and  identify  threats  is  only  half 
of  the  training  objective.  The  other  half  is 
to  learn  and  practice  tactical  maneuvers  and 
ECM  procedures  in  conjunction  with  other 
flight  mission  activities.  Integrating  an  EW 
training  device  with  an  operational  flight 
trainer  may  be  the  only  practical  means  of 
providing  adequate  tactical  engagement 
training  for  our  pilots  and  EWO's. 

The  interface  between  the  EW  simulator  and 
the  flight  trainer  can  vary  through  a wide 
spectrum,  depending  on  the  overall  extent  of 
the  simulation  and  training  requirements.  At 
a minimum,  the  EW  trainer  must  receive  from 
the  flight  trainer  computer  all  real-time 
flight  parameters  such  as  aircraft  position, 
altitude  and  attitude  in  order  to  adequately 
simulate  the  relative  power  and  incident 
angles  of  simulated  radar  signals  on  the 
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airborne  EW  antenna  system.  In  addition,  the 
EW  trainer  must  supply  to  the  flight 
simulator  computer  such  information  as  chaff 
salvo  drops  and  other  parameters  which  can 
affect  the  weight  and  balance  or  lift  and 
drag  coefficients  of  the  simulated  aircraft. 

Finally,  a principal  requirement  of  an  EW 
aircrew  trainee  is  flexibility  and 
updateability.  The  instructor  must  have  the 
capability  of  creating,  modifying  and 
controlling  a wide  variety  of  training 
exercises  and  tactical  encounters.  Ideally, 
the  mechanisms  for  controlling  the  simulated 
environment  should  be  simple  and  convenient, 
so  that  the  task  of  training  and  qualifying 
an  instructor  does  not  overshadow  the  student 
training  program.  Also,  the  EW  trainer 
should  accommodate  updates  in  EW  equipment 
and  EW  environments  with  minimal  impact  on 
cost,  development  time,  and  minimal  delays  to 
training  programs. 

SOME  DESIGN  CONSIDERATIONS 

There  are  numerous  factors  which  will  affect 
the  design  considerations  of  an  EW 
simulator/trainer  system.  These  design 
considerations  can  be  grouped  into  three 
major  categories:  the  EW  environment 
simulation,  the  EW  equipment  simulation,  and 
the  instructor  facilities  for  monitoring  the 
student's  responses  and  controlling  training 
exercises. 

Simulation  of  the  environment  involves  a 
modeling  of  each  emitter,  including  location 
and  signal  characteristics.  The  degree  of 
simulation  and  amount  of  hardware  and 
software  required  depends  heavily  upon  the 
type  of  EW  equipment  involved,  the  method  in 
which  the  EW  equipment  is  being  simulated, 
and  degree  to  which  the  equipment  is  being 
modeled.  A simplified,  low-cost  approach 
uses  "logical"  simulation,  in  which  a 
determination  is  made  as  to  whether  the 
signal  from  an  emitter  is  within  range  and 
line  of  sight.  If  so,  the  signal  is 
determined  to  be  detected  and  the  appropriate 
display  is  generated.  Unfortunately,  this 
"logical"  approach  typically  lacks  realism 
and  misses  important  training  cues.  At  the 
other  extreme  is  a pulse-by-pulse  simulation 
wherein  the  characteristics  of  each  pulse  are 
modeled  in  detail.  This  approach  requires 
excessive  hardware  and/or  software  to 
implement,  is  very  costly  and,  in  some  cases, 
oversimulates  the  inputs  to  the  simulated  EW 
equipment.  Depending  upon  the  specific 
application,  an  approach  lying  somewhere 
between  these  two  extremes  is  normally  used. 

The  EW  equipment  simulation  involves 
understanding,  in  detail,  the  operation  of 
the  various  EW  equipment  from  antenna  to 
display  and  readout.  Consider  the  antenna 


portion  of  an  EW  system.  This  may  be 
omnidirectional,  scanning,  multiple 
directional,  or  phased  array.  The  type 
incorporated  in  the  EW  equipment  has  a 
significant  effect  upon  the  method  of 
simulation.  For  example,  if  an  omni  is 
employed,  then  the  antenna  modeling  becomes 
greatly  simplified.  In  the  case  of  a 
multiple  monopulse  DF  array,  the  response  of 
each  individual  antenna  must  be  considered. 
In  some  cases,  an  azimuth-only  simulation  may 
be  sufficient;  in  others,  elevation  may  also 
be  required.  If  the  antennas  are  circularly 
polarized,  then  polarization  effects  need  not 
be  incorporated;  if  they  are  linearly 
polarized,  then  these  effects  need  to  be 
simulated.  Scanning  antennas  pose  a 
different  type  of  problem. 

The  receiver  simulation  involves  many  of  the 
same  tradeoffs  as  the  antennas.  For  example, 
the  receiver  may  be  wideband  and  capable  of 
receiving  multiple  signals  simultaneously,  or 
narrowband  to  select  only  one.  It  may  be  of 
a scanning  type  to  drive  a panoramic  display, 
or  tunable  to  signals  of  interest.  Further, 
the  tuning  may  be  automatic  or  manual.  These 
various  receiver  characteristics  must  be 
accurately  simulated  to  ensure  proper 
training  cues.  One  example  is  the  narrowband 
receiver  that  is  used  to  reduce  the 
environment  to  a single  signal  of  interest. 
No  receiver  possesses  the  ideal,  infinitely 
sharp  skirts  in  the  IF  processing  section; 
thus,  closely  spaced  signals  may  not  be 
separable. 


In  addition  to  EW  environment  and  EW 
equipment  simulation,  another  consideration 
involves  the  instructor  control  and 
monitoring  facilities.  This  area  allows  the 
most  freedom  for  the  simulator  designer  to 
select  different  approaches.  The  ultimate 
goal  is  to  provide  the  instructor  with  the 
greatest  amount  of  capability  in  defining 
scenarios,  making  dynamic  alterations  to  it 
as  the  EW  operator  is  training,  and  providing 
the  ability  to  monitor  the  status  of  the 
environment,  equipment  modes  and  student 
responses.  Probably  the  largest  constraint 
is  cost.  There  are  qranhic  devices  which, 
when  couoled  with  the  aporooriate  software 
can  display  all  tyoes  of  disnlay  symboloqy. 
Althouqh  there  is  qreat  flexibility  in  this 
area  and  the  potential  is  nearly  unlimited, 
it  offers  the  desiqner  one  of  the  most 
significant  desiqn  challenges  to  orovide  the 
maximum  amount  of  information  and  control 
capability  with  whatever  cost  constraints 
have  been  imoosed. 
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VARIOUS  DESIGN  APPROACHES 


There  are  a number  of  methods  by  which  an  EW 
simulator/trainer  can  be  implemented.  These 
range  from  various  methods  of  signal 
generation  and  the  use  of  actual  EW  equipment 
to  a total  software  modeling  of  the 
environment  and  the  equipment  itself.  This 
latter  method,  which  is  employed  exclusively 
in  flight  simulators,  has  not  been  employed 
for  EW  simulation  until  recently. 

Possibly  the  biggest  disadvantage  to 
generating  live  signals  and  injecting  them 
into  the  EW  system  is  the  cost  and  complexity 
of  the  signal  generation  hardware  required. 
In  the  event  the  EW  system  possesses  limited 
recognition  logic,  these  signals  can  be 
generated  with  a minimum  of  complexity. 
However,  in  most  modern  wideband  EW  systems, 
like  the  radar  warning  receiver  (RWR),  more 
and  more  use  is  being  made  of  all  modulation 
parameters  including  amplitude,  scan,  pulse 
frequency  modulation  (PFM)  type,  and  pulse 
group  spacings.  As  the  RWR  makes  more  use  of 
these  parameters,  more  care  must  be  taken  to 
generate  precise  replicas  of  the  actual 
signal  to  ensure  proper  EW  system 
performance.  Although  certainly  possible, 
the  cost  of  generating  and  mixing  these 
parameters  with  the  required  modulations  and 
accuracy  makes  this  method  less  and  less 
desirable  from  a cost  standpoint. 

The  software  modeling  approach  to  EW 
simulation  provides  the  greatest  amount  of 
flexibility  and  growth  potential  since 
virtually  any  EW  system  under  any  conditions 
can  be  simulated  by  the  addition  of  more 
software.  This  type  of  simulation  requires 
the  initial  investment  of  a computer,  memory 
and  some  peripherals  and  display  driving 
hardware.  Once  this  investment  has  been 
made,  it  can  be  expanded  via  additional 
software  at  a relatively  gradual  slope.  It 
should  be  noted  that  the  hardware  signal 
generation  approach  would  also  require  a 
computer  or  intelligent  processor  to  control 
the  real-time  signal  modulation  and  gating. 

In  addition  to  cost,  there  are  a number  of 
other  benefits  inherent  in  the  software 
modeling  approach.  These  include  increased 
realism,  flexibility,  repeatability  and 
system  rel iabi 1 ity  and  maintai nabi 1 ity.  The 
increased  realism  is  achieved  because  all 
factors  can  be  modeled  to  whatever  extent  is 
necessary  to  produce  displays  corresponding 
to  real  life.  Nearly  every  EW  system, 
whether  analog  or  computer-based,  has  some 
anomalies.  In  some  cases,  these  provide  the 
EW  operator  with  additional  information  that 
can  assist  in  identifying  threat  signals. 
Whatever  the  case  or  result,  the  EW  operator 
will  face  these  effects  in  a combat  situation 


and  must  be  trained  to  deal  with  them.  In 
terms  of  reliability  and  maintainability,  the 
software  modeling  approach  uses  far  fewer 
components,  and  those  components  are 
primarily  off-the-shelf  commercial  computer 
parts. 

Another  advantage  of  the  software  approach 
involves  the  ability  of  the  instructor  to 
induce  artificial  EW  equipment  failures  and 
monitor  the  student's  reaction.  Since  the  EW 
system  is  modeled  in  software,  this  feature 
is  easily  incorporated.  Finally,  the 
remaining  simulator  requirements  of  dense  and 
complex  environment  generation,  as  well  as 
instructor  flexibility,  can  be  readily 
accommodated. 

Possibly  the  most  difficult  portion  of  the 
software  modeling  approach  involves  the 
signal  analysis  and  display  portion  of  the 
equipment.  In  somes  cases,  the  analysis 
portion  of  the  equipment  is  rather 
rudimentary  involving  pulse  repetition 
frequency  (PRF)  filters  for  pulsed  signals  or 
manually  tuned  oscillators  for  CW  signals. 
In  these  types  of  systems  the  signal 
processing  simulation  is  straightforward,  but 
proper  generation  of  the  displays  and  audio 
can  become  quite  involved.  These  systems 
possessing  minimal  analysis  and  recognition 
capability  rely  heavily  upon  operator 
interpretation  of  the  various  subtleties  and 
anomalies  in  the  displays.  Thus,  it  is 
imperative  that  the  modeling  be  accurate  in 
terms  of  tones  and  warbles  on  the  audio  and 
flicker  and  jitter  on  the  displays. 

Many  of  the  newer  EW  systems  incorporate 
computers  into  the  equipment  and  utilize 
extensive  software  analysis  to  perform  the 
actual  signal  recognition  and  identification 
functions.  Since  the  analysis  is  performed 
by  a computer,  the  displays  associated  with 
this  type  of  system  are  typically  "synthetic" 
with  only  the  results  of  the  analysis  being 
displayed.  In  many  cases,  the  display  is  in 
the  form  of  al phanumerics  graphically 
depicting  the  signal  environment  or  tabular 
readouts.  The  simulation  of  the  processing 
is  the  most  difficult  task  in  this  type  of 
system,  since  it  is  mandatory  that  the 
operator  be  presented  with  the  results  which 
would  have  been  obtained  with  the  actjal 
. In  those  environments  where  the 
equipment  might  false-alarm  and  report  a 
"phantom"  signal,  fhe  simulator  must  "fail" 
in  the  same  manner.  Likewise,  the  simulator 
must  not  detect  those  signals  which  the 
actual  EW  system  might  miss  due  to  overload 
or  peculiar  signal  conditions. 

A SUCCESSFUL  LOW-COST  SOFTWARE  SIMULATION 
APPROACH 

Applied  Technology  has  recently  delivered  and 
installed  two  EW  aircrew  trainers  designated 
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RSI  (RHAW  Simulator  Interface)  for  pilot 
training  on  an  analog  radar  warning  system. 
For  the  previously  stated  reasons  of  realism, 
fault  modeling  and  cost,  these  simulators 
employ  a software  simulation  approach  and  use 
only  the  displays  and  indicators  of  the  EW 
system.  The  majority  of  the  simulator  logic 
resides  in  software,  and  only  minimal 
interface  circuitry  is  required  to  drive  the 
displays  and  generate  the  real-time  audio. 

The  simulator  design  goal  was  to  accurately 
model  all  normal  and  several  failure  modes  of 
operation  of  this  equipment,  to  provide  a 
realistic  simulation  of  a flexible, 
interactive  EW  environment.,  and  to  provide  a 
number  of  instructor  control  and  monitoring 
functions.  The  hardware  used  to  satisfy 
these  requirements  consists  of  four  elements: 


t General -purpose  microprogrammable  mini- 

computer 

• General-purpose  CRT/keyboard 

• General-purpose  teletype,  hardcopy,  and 
data  entry  terminal 

• Special  purpose,  ATI-built  instructor 
display  and  control  chassis  with 
repeater  displays  and  controls  of  the  EW 
simulated  system  instruments,  with  EW 
emitter  control  switches  and  indicators, 
and  with  trainer  controls  and  status 
indicators. 

The  radar  warning  system  that  is  simulated 
consists  of  a four-channel  monopulse 
direction  finding  system.  There  are  four 
directional  antennas  located  90  degrees  apart 
around  the  aircraft,  and  a receiver  located 
near  each  antenna.  The  receiver  contains 
crystal  detectors  and  compression  video 
amplifiers.  Incoming  pulses  from  an  emitter 
which  are  strong  enough  to  be  detected  are 
then  amplified.  The  outputs  of  the  video 
amplifiers  are  routed  to  an  analyzer  where 
the  four  quadrants  are  summed  and 
thresholded.  Whenever  this  threshold  is 
crossed,  a synthetic  pulse  is  generated  and 
applied  to  a bank  of  PRF  filters  which 
classify  the  pulse  for  signal  type. 
Depending  upon  which  filter  passes  the  pulse, 
a "strobe  code"  is  generated.  The  four 
amplitudes  from  the  four  receiver  channels 
are  applied  to  a CRT  indicator  which 
vectorially  combines  these  signals  and  draws 
a strobe.  The  vector  addition  of  the  four 
channels  causes  the  angle  at  which  the  strobe 
is  drawn  to  be  an  approximati on  of  the 
directi on-of-arri val  (DOA)  of  the  signal  and 
the  length  to  be  representative  of  the  signal 
strength.  The  strobe  code  from  the  PRF 
filter  bank  causes  a dotted,  dashed,  or  solid 
trace  to  be  drawn.  Further,  each  strobe  is 


applied  to  audio  circuitry  for  injection  into 
the  intercom,  and  each  PRF  filter  output  is 
applied  to  an  associated  lamp  driver  to  light 
a legend  on  a threat  display  unit  (TDU).  The 
end  result  of  this  processing  is  to  supply 
the  pilot  (or  EWO)  with  a CRT  display  and  a 
billboard  legend.  The  CRT  is  a polar  display 
with  coded  strobes  depicting  threat  type  and 
their  angle  and  length  depicting 
di recti on-of-arri val  and  received  power, 
respectively.  The  billboard  displays  those 
signal  types  which  are  presently  being 
received  as  well  as  providing  the  operator 
some  control  capability  over  the  system. 

In  addition  to  the  radar  warning  system, 
there  is  an  associated  guidance  receiver 
which  intercepts  from  the  ground  certain 
command  signals  which  are  directed  to  a 
surface-to-air  missile.  This  receiver 
detects  these  signals,  performs  a logical 
analysis  to  determine  the  status  of  this 
command,  and  outputs  various  pertinent  data 
to  the  RWR  analyzer.  The  analyzer  uses  this 
data  to  highlight  certain  signals  on  the 
indicator,  light  appropriate  legends  on  the 
TDU,  and  issue  audio  alert  tones  into  the 
pilot's  intercom. 

The  simulation  of  the  radar  warning  receiver 
and  guidance  receiver  is  primarily 
accomplished  by  software  with  minimal 
hardware.  Using  only  the  operational  CRT 
display  and  TDU,  plus  a CRT  driver  card  and  a 
general-purpose  pulse  generator  card,  all 
other  operational  hardware  was  modeled  in 
software.  The  quadrant  antenna  spatial  gain 
characteristics,  the  receiver  transfer 
functions,  and  all  the  internal  analyzer 
logic  was  modeled  in  real-time  software.  The 
CRT  signals  and  audio  tones  produced  by  this 
simulation  technique  turned  out  to  be  highly 
realistic  with  respect  to  visual  cue 
accuracy,  correlation  between  audio  and 
visual  cues,  and  ability  to  repeat  most  of 
the  operational  equipment  anomalies. 

The  simulated  EW  environment  used  in  the  RSI 
is  designed  to  handle  16  simultaneously 
active,  yet  independently  controllable, 
threat  emitters.  A table  of  radar  parameters 
and  positional  information  is  assigned  to 
each  simulated  emitter.  Some  of  the  threat 
radars  which  can  generate  multiple  emissions, 
such  as  dual- beam  target  trackinq  radars  and 
SAM  tracking  radars  with  associated  guidance 
signals,  only  require  a single  emitter  by  the 
simulator.  Thus,  all  16  emitters  can  be 
assigned  to  different  targets.  At  any  moment 
in  a training  session  a tactical  scenario 
consisting  of  16  simultaneous  emitters  is 
quite  adequate.  However,  over  the  entire 
training  session  many  sets  of  tactical 
scenarios  are  required.  The  RSI  provides 
four  scenarios  as  a standard  feature  which  is 
easily  expandable.  With  four  scenarios 
containing  16  emitters  each,  the  RSI  provides 
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storage  and  simulation  of  64  unique  threat 
emitters.  Each  scenario  Is  assigned  a 
geographic  location  within  the  gaming  area  of 
the  flight  simulator.  The  16  emitters  which 
can  be  assigned  to  a scenario  can  be 
positioned  either  Inside  or  outside  the 
scenario's  geographic  boundaries,  provided 
the  emitter  position  is  somewhere  within  the 
overall  gaming  area.  Scenarios  can  be  of  a 
different  size,  and  overlapping  is  permitted. 
When  the  instructor  selects  the  "tactical 
situation  display"  on  his  CRT  terminal,  the 
current  scenario  of  emitters  is  used  for 
display.  Scenarios  are  automatically 
switched  and  updated  on  the  "tactical 
situation  display"  whenever  the  simulated 
aircraft  flies  from  one  scenario  boundary 
into  another.  To  account  for  overlapping 
scenarios,  each  scenario  is  assigned  a 
priority  number.  The  highest  priority 
scenario  is  selected  for  display  if  a 
conflict  exists. 

In  conjunction  with  instructor  display 
functions,  the  emitters  of  the  currently 
active  scenario  are  assigned  to  the  control 
switch  on  the  instructor  display  and  control 
chassis.  For  each  emitter,  the  instructor 
can  select  either  an  automatic  or  manual 
mode.  In  the  automatic  mode,  four  range 
rings  for  each  emitter  are  preprogrammed. 
The  outermost  ring  represents  the  maximum 
search  range  for  the  emitter;  the  next  one 
represents  the  tracking  range.  As  the 
aircraft  passes  through  the  first  range  ring, 
the  emitter  is  automatically  activated  and 
placed  in  the  search  mode.  When  it  passes 
into  the  next  ring,  it  is  placed  in  the  track 
mode.  The  last  two  range  rings  are  used  for 
surface-to-air  missile  (SAM)  sites.  When  the 
third  boundary  is  crossed,  the  guidance 
signal  is  activated  and  placed  in  a 
non-launch  condition.  Finally,  when  the  last 
boundary  is  crossed,  the  missile  is  launched 
and  the  appropriate  missile  launch  condition 
is  generated.  In  addition  to  this  automatic 
emitter  mode  control,  the  instructor  can 
select  the  manual  mode  of  operation.  In  this 
mode,  the  instructor  can  override  the 
automatic  emitter  mode  and  select  any  of  the 
modes  (search,  track,  missile  active,  or 
launch  conditions)  desired.  With  this 
capability,  the  instructor  can  simulate  an 
optical  tracking  site. 

The  RSI  simulator  has  two  basic  modes  of 
operation:  a mode  integrated  with  the  flight 
simulator  and  a stand-alone  mode.  In  the 
integrated  mode,  the  simulator  receives  the 
dynamically  updated  position  and  attitude  of 
the  aircraft  from  the  flight  simulator  and 
uses  this  data  in  subsequent  computations  for 
determining  the  appropriate  display  and  audio 
indications.  Measurement  of  the  aircraft 
parameters  and  computation  of  ranges  and 
received  powers  is  made  ten  times  per  second 


to  permit  smoothly  varying  displays.  In  this 
mode,  the  pilot  sees  and  hears  the 
appropriate  indications  as  a function  of  his 
flight  path.  Emitters  move  in  angle  on  the 
indicator,  and  their  relative  powers  vary 
appropriately.  In  addition,  the  simulator 
provides  automatic  threat  status  changes  as  a 
function  of  slant  range  and  line  of  sight. 

The  stand-alone  mode  is  used  primarily  for 
classroom  training  with  one  or  more  students. 
In  this  mode,  the  aircraft  can  be  flown  by 
the  instructor  via  typed  commands  on  the  CRT 
terminal  to  control  altitude,  ground  speed, 
and  heading.  Alternatively,  a prerecorded 
tape  can  be  used  from  a previously  simulated 
mission.  In  either  case,  the  controls  on  the 
threat  display  unit  of  the  EW  system  are 
fully  operative  and  can  be  used  to 
demonstrate  pertinent  EW  system  features. 
Further,  the  system  can  be  placed  in  a 
"freeze"  mode  or  backed  up  in  time  to 
demonstrate  critical  display  characteristics 
or  signal  parameters. 

The  CRT  terminal  and  instructor  control  panel 
operates  in  both  the  integrated  and 
stand-alone  modes,  providing  the  primary 
means  for  the  instructor  to  control  the  RSI 
simulator.  It  provides  a comprehensive  set 
of  commands  to  enable  alteration  and 
monitoring  of  emitter  scenarios  and  emitter 
parameters,  as  well  as  to  monitor,  insert,  or 
delete  simulated  equipment  malfunctions.  The 
CRT  monitor  provides  five  separate  pages  of 
information  to  enable  effective  monitoring  of 
the  exercise  status.  The  contents  of  each 
page  are  summarized  below: 

• First  is  an  index  page,  which  indicates 
which  pages  are  available  for  viewing 
and  the  commands  required  to  call  each 
one. 

• Second  is  the  emitter  scenario  page, 
which  displays  detailed  status  on  each 
of  the  scenarios.  On  it  are  the 
scenario  coordinates  and  scaling 
factors.  Further,  it  contains  summary 
information  such  as  location  and  type  of 
each  of  the  emitters  assigned  to  each 
scenario. 

• Third  is  the  emitter  parameter  page, 
which  contains  all  of  the  detailed 
information  on  each  emitter.  Contained 
on  this  page  is  the  transmission  power, 
RF  frequency,  PRF,  pulsewidth,  scan 
characteristics,  and  other  parameters 
defining  each  simulated  threat  radar. 
Further,  it  contains  the  ranges  at  which 
the  signal  changes  modes  ( i . e . , search 
to  track)  should  the  automatic  mode 
change  be  selected  for  that  emitter. 
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• The  fourth  page  contains  the  tactical 
situation  display  (one  for  each  of  the 
scenarios).  The  left  half  of  this  page 
contains  summary  information  on  the 
scenario  coordinates,  site  and  emitter 
types.  Also,  it  contains  current 
aircraft  headinq,  speed  and  altitude, 
which  are  automatically  undated.  On  the 
riqht  half  of  the  page  is  a graphical 
representation  of  the  tactical 
situation.  This  tactical  situation 
display  is  the  paqe  normally  selected 
durinq  the  traininq  session;  it  is  used 
to  indicate  graphically  to  the 
instructor  the  aircraft  location  with 
respect  to  the  emitters  so  that  the 
instructor  can  manually  activate  and 
deactivate  sites  and  attempt  to  confuse 
or  lure  the  pilot  into  lethal 
situations. 

• Fifth  is  the  malfunction  page,  which 
enables  the  instructor  to  determine  the 
status  of  simulated  EW  system 
malfunctions.  F»ch  malfunction  can  be 
independently  turned  on  or  off  at  any 
time  during  the  training  session  to  test 
the  reactions  of  the  pilot. 

Although  the  various  modes  of  operation  and 
instructor  flexibility  are  important,  it  is 
the  realism  of  the  simulator  that  makes  it 
truly  unique.  This  realism  is  due  primarily 
to  the  detailed  modeling  of  the  EW  system 
anomalies.  These  anomalies  appear  in  both 
audio  and  visual  indications.  Each  EW  system 
has  its  own  set  of  anomalies,  and  those 

programmed  in  the  RSI  software  are  designed 
for  the  particular  radar  warning  system  being 
simulated. 

The  most  recognized  of  these  anomalies  is  the 
"fanning  effect"  of  strobes  from  a single 
emitter.  Briefly,  this  effect  is  due  to 
antenna  patterns,  receiver  transfer 

functions,  pulse  filter  characteristics  and 
the  vector  addition  used  to  compute 
direction-of-arrival . The  net  result  is 
that  amplitude  variations  of  a received 
signal  result  in  apparent  angle  changes. 
Thus,  any  emitter  exhibiting  amplitude 

modulation  due  to  scanning  has  some  angular 
fanning  on  the  radar  warning  display. 
Furthermore,  since  the  antennas  are  not 
perfectly  circularly  polarized,  but  are 

slightly  elliptical,  polarization  effects 
cause  an  effective  amplitude  modulation  to  be 
received  and,  hence,  fanning  to  be  produced. 

These  effects  are  not  particularly 
detrimental  to  the  effectiveness  of  the  radar 
warning  system,  and  in  many  cases  are  used  to 
advantage.  For  example,  there  is  one  threat 
signal  which  employs  two  beams  to  locate  the 
target  in  azimuth  and  elevation.  The  two 
beams  are  not  synchronized  and  hence  they 


tend  to  "walk  through"  each  other.  During 
these  walkthrough  periods,  a greater  amount 
of  energy  is  received  and  the  fanning 
decreases.  Between  walkthrough  periods,  the 
fanning  increases  and  a lesser  amount  of 
energy  is  received.  In  the  case  of  another 
dual-beam  threat  signal,  the  beams  are 
synchronized  and  this  additional  fanning 
effect  is  not  observed.  A trained  pilot  can 
quite  readily  distinguish  these  two  signals 
and  make  a precise  identification  of  the 
threat  emitters  over  and  above  that  provided 
by  the  analysis  logic  in  the  warning  system. 

In  another  case,  there  are  two  different 
threat  signals  in  the  same  RF  frequency  band 
and  with  the  same  PRF  characteristics.  The 
only  distinguishing  characteristic  between 
them  is  their  scan  characteristic.  One  of 
the  signals  operates  in  a lobe-on-receive- 
only  (LORO)  mode,  and  this  is  extremely 
constant  in  terms  of  amplitude  variation. 
The  other  signal  operates  in  a conical  scan 
mode  and  therefore  exhibits  a sinusoidal 
amplitude  modulation.  Although  these 
modulations  are  small  and  are  difficult  to 
see  as  strobe  length  changes  on  the  azimuth 
indicator,  the  fanning  effects  cause  this 
modulation  to  be  highlighted  and,  hence, 
quite  readily  discerned  by  a trained  pilot. 

The  RSI  software  incorporates  these  effects 
and  has  most  successfully  passed  extreme 
scrutiny  from  experienced  test  pilots 
intimately  familiar  with  the  radar  warning 
system.  Briefly,  the  algorithms  which 
simulate  these  effects  utilize  scan  type, 
range,  beamwidths,  and  other  pertinent 
factors  to  describe,  on  a pulse-to-pulse 
basis,  the  strobe  display  that  would  be 
generated. 


The  RSI  software  is  organized  as  two  major 
program  loops,  each  operating  at  a different 
speed.  The  outer  loop,  executed  at  a rate  of 
ten  times  per  second,  contains  the  major 
simulation  programs.  When  it  is  entered,  the 
aircraft  coordinates  are  entered  into  the 
processor  and  scaled.  First,  the  elements  of 
an  Euler  matrix  are  computed  and  used  to 
perform  an  inertial-to-body  axis 
transformation.  Next,  the  range  and  power 
from  each  emitter  are  calculated.  Also,  the 
angles  the  incident  energy  makes  with  each 
antenna  are  computed.  A power  calculation  is 
made,  and  the  appropriate  antenna  gain  for 
that  azimuth  and  elevation  is  used.  Next, 
the  receiver  transfer  characteristics  are 
modeled,  and  finally  a threshold  check  is 
made  to  determine  if  the  received  pulse  was 
above  the  receiver  threshold.  If  so, 
additional  modeling  to  determine  strobe  code 
and  fanning  characteristics  is  made.  These 
characteristics  are  then  used  for  the  next 
one-tenth  second  until  a new  computation  is 
made. 
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The  inner  loop  operates  on  a pulse-to-pulse 
basis  and  is  implemented  entirely  in 
microcode.  Briefly,  it  is  activated  by  an 
interrupt  at  the  conclusion  of  each  strobe 
generation.  It  determines  which  pulse  is 
eligible  next  for  display,  addresses  through 
the  appropriate  tables  to  locate  the  next 
angle,  amplitude  and  strobe  code  and  outputs 
it  to  the  interface  circuitry  which  draws  the 
next  strobe.  The  microcode  program  requires 
approximately  20  microseconds  to  execute. 
Since  a strobe  requires  100  microseconds  to 
draw,  it  is  impossible  to  saturate  the 
simulator  regardless  of  pulse  density  or 
number  of  emitters.  At  most,  it  requires  20 
percent  of  the  CPU  time.  Alternate  designs 
have  hardware  complexities  orders  of 
magnitude  greater.  The  total  hardware  for 
this  approach,  without  the  deflection 
amplifiers,  is  physically  located  on  two  I/O 
board  and  requires  less  than  100  integrated 
circuits. 

FUTURE  TRAINING  DEVICE  REQUIREMENTS  AND 
SOLUTIONS 

Looking  into  the  future,  it  is  obvious  that 
the  EW  environment  will  increase  in 
complexity  and  density.  Aircraft  EW 
equipment  will  also  increase  in  complexity  to 
keep  pace  with  the  growing  environment.  A 
look  at  the  ALR-67  and  other  radar  warning 
and  power  management  systems  illustrates  this 
increasing  complexity.  These  new-generation 
onboard  EW  systems  must  interface  with  other 
aircraft  avionic  systems,  control  active  ECM 
equipment  and,  in  some  cases,  interface  to 
weapon  delivery  systems.  With  the  expansion 
in  the  use  of  processors  and  microprocessors, 
the  tasks  performed  by  the  EW  systems  are 
taking  quantum  jumps.  The  next  generation  of 
EW  systems  will  include  IFM's,  laser 
intercept  systems,  and  new  onboard  jamming 
systems.  All  this  points  to  an  increased 
burden  on  our  pilots  and  EWO's.  The  only 
solution  for  these  EW  personnel  to  acquire 
and  maintain  proficiency  is  through  training, 
and  the  only  practical  and  cost-effective 
means  of  training  is  through  ground-based  EW 
equipment  trainers. 


EW  simulators  are  still  at  the  formative 
stage  and  a number  of  significant  advances 
can  be  expected  in  the  future.  Applied 
Technology  believes  the  RSI  simulator  has 
demonstrated  two  such  advances.  The  first  is 
in  the  area  of  realism.  The  algorithms 
employed  in  this  simulator  provide  a detailed 
modeling  of  the  environment  as  well  as  the 
equipment.  The  end  result  is  a display  and 
audio  tones  which  have  undergone  the  final 
test  of  experience.  The  pilots  who  have  seen 
and  heard  the  simulator  believe  it  to  be  a 
most  accurate  representation  of  the  real 
thing.  Side-by-side  comparisons  with  signal 
generators  and  the  actual  EW  system  show  the 
simulator  to  be  far  superior  to  even  an 
untrained  eye  let  alone  one  of  experience. 
Second,  the  use  of  microprogramming  to 
eliminate  pulse  and  scan  generators  has  made 
a most  dramatic  reduction  in  system 
complexity  and  resultant  cost.  Further,  it 
allows  a far  greater  degree  of  flexibility  to 
incorporate  new  systems  and  new  capabilities. 

Next-generation  EW  simulators  must  include 
these  new  capabilities  and  must  meet  the 
realism  standards  which  are  within  the  realm 
of  state-of-the-art  technology.  I he 
requirements  are  real  and  there  is  no  reason 
they  cannot  be  met  in  a cost-effective 
manner.  Further,  these  simulators  should 
include  SAM  missile  dynamics  and  provide 
automatic  scoring  of  the  Dilot's  reactions. 
Flight  simulators  can  be  used  to  determine 
the  aerodynamic  effects  when  new  ordnance  or 
equipment  is  carried  or  modifications  to  the 
airframe  are  made.  The  flight  simulators 
have  modeled  the  aerodynamic  effects  so 
completely  that  these  alterations  are  tested 
on  the  simulator  prior  to  being  flown.  With 
sufficient  realism,  similar  tests  could  be 
made  with  tomorrow's  EW  simulator  to 
determine  the  effectivity  of  new  maneuvers, 
tactics,  and  equipment. 

We  believe  that  EW  expertise  is  the  key  to 
meeting  the  EW  simulation  requirements  of  the 
future.  Realism  is  the  key  to  effective 
training,  and  an  intimate  understanding  of 
the  equipment  is  vital  to  developing  a 
realistic  simulation. 


ABOUT  THE  AUTHORS 


MR.  BURTON  ID.  GRAVES  is  an  Engineering  Manager  at  Applied  Technology,  Divi- 
sion of,  Ttek  Corporation.  His  responsibilities  include  development  of  signal 
processing  techniques  for  the  detection,  recognition,  and  identification  of 
various  radar  signals  and  all  software  development  in  the  division.  He  has 
been  involved  uiith  the  system  design  and  software  development  efforts  on 
nearly  all  ATT  computer-based  radar  warning,  pouter  management,  and  simulator 
systems.  Mr.  Graves  holds  B.S.  and  M.S.  degrees  in  electrical  engineering 
from  the  California  Institute  of  Technology. 

MR.  DANIEL  J.  MARCHEGIANI  is  a Programming  Specialist  at  Applied  Technology, 
Division  of  Ttek  Corporation.  His  responsibilities  have  involved  system 
design  and  software  development  on  a number  of  computer-based  pouter  manage- 
ment and  simulator  systems.  He  has  been  involved  uiith  the  RHAW  Simulator 
Tnterface  (RSI)  simulator  project  since  its  inception  and  has  provided  the 
majority  of  the  system  and  software  design.  He  has  been  the  software  lead 
engineer  and  assistant  program  manager  on  this  program.  Mr.  Marchegiani 
holds  a B.S.  degree  in  aeronautical  engineering  from  Rensselaer  Polytechnic 
Tnstitute  and  a M.S.  degree  in  aerospace  engineering  from  Cornell  University. 


SUITABILITY- FOR-TRAINING  EVALUATION 
OF  THE  CH-47  FLIGHT  SIMULATOR 

DR.  GARVIN  L.  HOLMAN 
US  Amy  Research  Institute  Field  Unit 
Fort  Rucker,  Alabama 


BACKGROUND 

Although  many  Improvements  have  been 
made  In  the  conduct  of  US  Amy  helicopter 
flight  training,  the  most  Important  part  of 
the  student's  Instruction  is  still  performed 
In  an  aircraft  under  the  direct  supervision 
of  an  Instructor  pilot.  This  method  is 
extremely  costly  in  terms  of  time  required 
on  the  flight  line  by  both  student  and 
instructor  and  in  terms  of  flying  hour  costs 
in  today's  sophisticated  aircraft. 

These  costs  became  more  apparent  during 
the  late  1 960 ' s when  the  Army  experienced 
a rapid  expansion  of  its  aviation  capability. 
The  huge  increase  in  the  cost  of  aviation 
training  which  accompanied  this  period  of 
expansion  clearly  indicated  the  need  for 
economical  synthetic  flight  training  systems 
which  could  reduce  the  requirement  for  use 
of  operational  helicopters. 

To  fulfill  this  need,  the  Army  approved 
a Qualitative  Materiel  Requirement  for 
development  of  a synthetic  flight  training 
system  in  July  1967.  Eventually,  a contract 
was  awarded  in  June  1973  for  the  construc- 
tion of  a flight  simulator  of  the  CH-47C 
Chinook  aircraft. 


o Slx-degree-of-freedom  cockpit  motion 
system. 

o Visual  simulation  system  consisting 
of: 

Camera-model  image  generation 
system 

Synthetic  terrain  and  ground 
symbol  generator 

Infinity-image  display  systems 
mounted  on  cockpit. 

o Digital  computation  system  and 
associated  software. 

These  subsystems  will  be  described  in  the 
subsequent  paragraphs. 

FLIGHT  C0M>ARTMENT 

The  operational  flight  trainer's  flight 
compartment  interior  contains  a replica  of 
the  pilot's  and  copilot's  positions,  the 
i ns tructor/opera tor's  station,  and  an 
observer's  position.  A doorway  through  the 
rear  bulkhead  provides  for  entry  into  the 
compartment. 


The  operational  test  of  the  CH47FS  began 
in  January  1977  and  was  completed  in  August 
1977.  The  training  effectiveness  studies 
were  conducted  by  the  US  Army  Aviation  Board 
and  the  US  Army  Research  Institute  Field 
Unit  at  Fort  Rucker,  Alabama. 

DESCRIPTION 

The  CH-47  flight  simulator  (CH47FS)  is 
a ground-based  flight  simulator  designed  to 
provide  training  on  CH-47C  helicopter  cock- 
pit procedures,  aircraft  control,  contact 
maneuvers,  emergency  procedures,  sling  load, 
confined  area  operations,  and  instrument 
flight  procedures. 

The  simulator  system  consists  of  the 
following  major  subsystems: 

o Flight  compartment  consisting  of: 

Trainee  station  for  pilot  and 
copilot  (cockpit) 

Instructor/operator  station 


The  trainee  station  is  located  in  the 
forward  portion  of  the  flight  compartment  and 
is  a replica  of  the  pilot's  and  copilot's 
positions  forward  of  their  seat  backs.  The 
controls,  indicators,  and  panels  operate  and 
have  the  same  appearance  as  those  in  the  air- 
craft. The  aft  portion  of  the  center  console 
contains  the  instructor  pilot/trainee  problem 
control  panel.  The  trainees'  seats  are  vi- 
brated individually  to  simulate  the  continuous 
and  periodic  oscillations  and  vibrations 
experienced  by  the  crew  during  normal  and 
emergency  flight  conditions  and  maneuvers, 
including  vibrations  representing  progressive 
malfunctions.  Four  loudspeakers  provide  aural 
cue  sounds  with  characteristics  of  location, 
frequency,  and  amplitude  simulated  within 
limits  of  safety. 

The  instructor  station  is  located  in  the 
instructor  area,  aft  of  the  cockpit,  in  the 
flight  compartment.  It  provides  information 
and  controls  with  which  the  instructor  can 
effectively  monitor  and  evaluate  student 
performance  and  control  the  training  problem. 
The  controls  are  located  on  a sloping  control 
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panel  below  two  cathode-ray  tube  (CRT)  dis- 
plays which  are  mounted  side  by  side  with 
their  longer  display  surface  dimension  verti- 
cal. The  instructor  is  also  able  to  operate 
from  either  trainee  station  by  using  the 
problem  control  panel  on  the  center  console. 

MOTION  SUBSYSTEM 

The  flight  compartment  is  mounted  on  a 
six-degree -of- freedom  motion  subsystem  con- 
sisting of  a moving  platform  assembly  driven 
and  supported  from  below  by  six  identical  48- 
inch  hydraulic  actuators.  Simulation  includes 
the  motion  due  to  changes  in  aircraft  attitude 
as  a result  of  flight  control  and  the  rotor 
operation,  rough  air  and  wind,  changes  in  air- 
craft weight  and  center  of  gravity,  as  well  as 
effects  of  buffet,  blade  stall,  blade  imbal- 
ance, blades  out  of  track,  and  touchdown 
impact. 

The  simulation  program  causes  the  motion 
subsystem  to  respond  to  aerodynamic  forces 
and  moments  within  the  mechanical  limits  of 
the  system.  All  motions  except  pitch  are 
imperceptibly  washed  out  to  the  neutral  posi- 
tion after  the  computed  accelerations  have 
reached  zero.  Pitch  attitude  is  maintained 
as  necessary  to  simulate  sustained  longitu- 
dinal acceleration  cues.  Acceleration  onset 
cues  are  scaled  as  large  as  possible  to  fully 
utilize  the  motion  capabilities  of  each  de- 
gree of  freedom. 

VISUAL  SUBSYSTEM 

The  visual  subsystem  is  a camera-model 
system  which  provides  a full  color  television 
image  in  the  forward-looking  window  displays. 
The  chin  window  displays  are  provided  by  a 
synthetic  terrain  generator. 

The  camera-model  visual  system  con- 
sists of  a 24  x 56  foot  three-dimensional 
terrain  model  viewed  by  a television  camera 
and  optical  probe  mounted  on  a movable 
gantry.  Servomechanisms  on  the  gantry  posi- 
tion the  camera  and  probe  in  accordance  with 
the  position  and  attitude  of  the  simulated 
aircraft.  The  model  board  is  mounted  verti- 
cally to  minimize  floor  space  requirements 
and  has  scales  of  1:400  and  1:1500.  The 
1:400  scale  model  is  a replica  of  the  east 
side  of  Hanchey  Army  Heliport  and  represents 
an  area  approximately  0.5  x 1.6  nautical 
miles  (nm).  It  simulates  the  detail  needed 
for  taxi  work  and  low-altitude  hovering.  The 
1:1500  scale  board  contains  Goldberg  stage- 
field  and  the  surrounding  terrain  as  would 
be  seen  in  southeast  Alabama  during  spring. 

The  gaming  area  of  the  1:1500  scale  model 
represents  5.75  x 11.75  nm  and  is  used  for 
the  training  of  tasks  requiring  a larger 
geographical  area  such  as  general  airwork, 


contact  maneuvers,  pinnacle,  confined  area, 
and  sling  load  operations.  A special  effects 
generator  provides  sky,  cloud,  haze,  and 
limited  visibility  effects  into  the  displayed 
scene  under  the  control  of  the  instructor. 

Day,  dusk,  and  night-light  conditions  are  also 
simulated. 

The  synthetic  terrain  generator  used  for 
the  pilot's  and  copilot's  chin  window  displays 
provide  a special  ground  symbol  and  a terrain 
representation  consisting  of  a regular  checker- 
board pattern  of  alternating  7-foot  green  and 
brown  squares,  in  correct  perspective  for  each 
trainee  eyepoint.  These  squares  form  a larger 
square  pattern  56  squares  on  a side  with  con- 
tinuous green  terrain  beyond  the  squares  and 
white  sky  beyond  the  horizon.  When  the  simu- 
lated helicopter  is  within  a 0.7  nautical  mile 
radius  of  a symbol  location,  and  between  10 
and  200  feet  above  the  ground  reference  plane, 
the  terrain  pattern  is  automatically  computed 
and  displayed;  beyond  this  range  the  synthetic 
terrain  display  is  either  in  or  above  clouds. 

The  visual  images  generated  by  the  camera- 
model  and  synthetic  terrain  systems  are  dis- 
played via  closed-circuit  television  to  both 
the  pilot  and  copilot  simultaneously  in  their 
forward  and  chin  window  displays.  The  entire 
display  is  collimated  by  a beam  splitter  and 
curved  mirror.  The  total  field  of  view  visi- 
ble by  movement  of  the  head  is  approximately 
48  degrees  horizontal  and  36  degrees  vertical 
on  the  forward  windows,  and  45  degrees  down 
and  25  degrees  outboard  in  the  chin  windows. 

COMPUTATION  SUBSYSTEM 

The  computation  subsystem  consists  of  a 
dual  PDP  11/45  computer  with  associated  memory 
and  peripheral  units.  The  operational  soft- 
ware consists  of  an  executive  program  and  real- 
time simulation  programs.  The  real-time  simu- 
lation programs,  in  conjunction  with  the 
appropriate  hardware,  provide  simulation  of 
flight  performance,  power  plants  and  engine- 
related  systems,  aircraft  accessory  systems, 
radio  communication  and  navigation  equipment, 
atmospheric  conditions,  flight  control  systems, 
and  malfunctions.  The  executive  program 
includes  computer  diagnostics,  a daily  opera- 
tional readiness  check  program,  and  a test 
exercise  program. 

EVALUATION 

GENERAL  METHOD 

The  training  effectiveness  of  the  CH47FS 
was  evaluated  in  two  studies:  one  to  deter- 
mine the  transfer  of  training  between  the 
simulator  and  the  aircraft  in  an  institutional 
setting  and  another  to  determine  its  effec- 
tiveness in  maintaining  or  increasing  flying 
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skills  in  an  operational  setting.  Study  I was 
of  classical  two  group  training  design  using 
aviators  undergoing  transition  training  to  the 
CH-47.  Study  II  assessed  the  training  bene- 
fits of  periodic  training  of  operational  CH-47 
aviators  in  the  CH47FS. 

STUDY  I 

INSTITUTIONAL  TRAINING 

SUBJECTS 

The  experimental  group  consisted  of  24 
student  aviators  selected  four  at  a time  from 
each  of  the  six  CH-47  transition  classes  that 
participated  in  the  study.  The  experimental 
aviators  were  selected  on  the  basis  of  two 
screening  tests  as  well  as  recent  and  total 
flight  experience  to  be  equivalent  to  the 
remainder  of  the  class.  Those  35  Army  avia- 
tors in  the  six  classes  not  chosen  as  experi- 
mental subjects  were  the  control  group  avia- 
tors. 

INSTRUCTOR  PILOTS 

In  preparation  of  these  training  studies, 
six  CH-47  instructor  pilots  (IPs)  spent  three 
weeks  learning  to  operate  the  simulator  and 
practicing  teaching  in  the  device.  Four  of 
the  IPs  were  from  the  US  Army  Aviation  Center 
and  were  engaged  in  teaching  CH-47  transition 
courses.  Two  of  the  IPs  were  from  Forces 
Conmand  (FORSCOM)  units  and  were  engaged  in 
combat  readiness  flying  (CRF)  training. 

INDEPENDENT  VARIABLE 

The  independent  variable  in  this  experi- 
ment was  the  use  of  the  CH47FS  in  the  course 
of  instruction  (COI)  of  CH-47  transition 
training.  The  control  group  was  trained  to 
fly  the  CH-47  in  a COI  developed  over  several 
years  with  a nominal  flying  time  of  30  hours 
(US  Army,  1975).  The  COI  is  split  into  two 
phases  of  approximately  15  hours  each;  a 
basic  phase  and  an  advanced  phase.  In  the 
basic  phase  the  following  procedures  and 
maneuvers  are  taught:  pre-  and  postflight 
inspections,  cockpit  procedures,  taxiing, 
hovering,  various  takeoff  and  landing 
maneuvers,  general  airwork,  and  emergency 
procedures.  The  advanced  phase  includes 
training  in  confined  area  and  pinnacle  opera- 
tions, external  and  internal  load  operations, 
slope  operations,  water  operations,  and  emer- 
gency procedures.  Between  the  basic  and  ad- 
vanced phases  there  is  a checkride  and,  at 
the  end  of  the  advanced  stage,  there  is  the 
final  aircraft  qualification  checkride. 

The  COI  used  by  the  experimental  group 
was  the  same  as  that  used  by  the  control  group 
except  that  the  instruction  was  conducted  in 


the  CH47FS  rather  than  in  the  aircraft.  Due 
to  design  limitations,  internal  load,  slope 
and  water  operations  could  not  be  performed 
in  the  simulator.  The  checkride  given  between 
phases  was  accomplished  in  the  simulator  and 
again  in  the  aircraft.  This  basic  phase  check- 
ride was  the  first  time  the  experimental  avia- 
tors had  flown  in  the  CH-47  aircraft.  The 
advanced  phase  of  training  was  then  conducted 
in  the  simulator  and  the  final  checkride  given 
in  the  simulator.  This  was  followed  by  an 
identical  checkride  in  the  aircraft.  Those 
maneuvers  not  performed  satisfactorily  on  the 
last  checkride  were  then  trained  in  the  air- 
craft and  the  CH-47  qualification  checkride 
given.  Thus,  the  experimental  group  took 
three  more  checkrides  than  the  control  group. 

PERFORMANCE  MEASURES 

The  CH-47  transition  course  was  divided 
into  32  separate  and  gradable  tasks  or  maneu- 
vers. During  daily  training,  each  time  a 
maneuver  was  performed,  the  IP  recorded  that 
performance  on  a daily  grade  sheet  designed 
for  this  study.  For  each  maneuver  the  IP 
recorded  an  evaluation  of  that  maneuver  on  a 
twelve  point  scale,  the  time  spent  on  the 
maneuver,  and  a rating  of  subtasks  associated 
with  the  maneuver. 

The  twelve  point  scale  used  to  evaluate 
each  maneuver  was  based  on  a scale  by  Reid 
(1975).  The  rating  scale  actually  encompassed 
thirteen  points  since  a maneuver  demonstrated 
by  the  IP  was  coded  as  a zero.  Varying  de- 
grees of  "unsatisfactory"  performance  were 
rated  one  through  three,  "fair"  was  rated 
four  through  six,  "good"  was  rated  seven 
through  nine,  and  "excellent"  was  rated  ten 
through  twelve.  Performance  level  six  was 
considered  the  minimal  acceptable  level  of 
skill  and  was  the  criterion  level  towards 
which  training  was  aimed. 

Thirty  of  the  32  graded  maneuvers  were 
also  divided  into  subtasks  and  the  performance 
of  these  subtasks  was  evaluated.  Each  subtask 
was  rated  on  the  basis  of  performance  being 
(1)  near  perfect,  (2)  error  present  but  with- 
in acceptable  standards,  (3)  error  present  and 
beyond  acceptable  standards.  The  acceptable 
standards  were  defined  by  the  Flight  Training 
Guide  and  were  well  known  to  the  IPs  and 
students. 

CUMULATIVE  TRANSFER  EFFECTIVENESS  RATIOS 

The  most  important  descriptive  statistic 
for  the  evaluation  of  the  training  effective- 
ness of  the  CH47FS  was  the  cumulative  transfer 
effectiveness  ratio  (CTER).  The  CTER,  as 
described  by  Roscoe  (1971  , 1972,  1973),  is 
a measure  of  the  savings  realized  in  learn- 
ing to  operate  an  aircraft  by  first  training 
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in  a training  device.  The  formula  for  the 
CTER  based  on  training  trials  is: 

A/C  trials  control  group  - A/C  trials  exp  group 
Simulator  trials  exp  group 

CTERs  were  calculated  on  the  basis  of  trials 
for  each  individual  maneuver  in  the  COI.  A 
CTER  is  a measure  of  the  training  efficiency 
of  a training  device.  A ratio  of  1.0  indi- 
cates the  training  device  is  as  efficient  as 
the  actual  device,  a ratio  greater  than  1.0 
indicates  the  training  device  is  more  effi- 
cient, and  a ratio  of  less  than  1.0  indicates 
that  the  training  device  is  less  efficient. 

LEARNING  CURVES 


Learning  curves  are  graphical  presenta- 
tions of  changes  in  skill  that  occur  with 
practice  or  overtime.  The  learning  curves 
in  this  paper  relate  the  skills  of  the  avia- 
tors on  each  maneuver  (as  measured  by  their 
IPs  on  the  twelve-point  scale)  to  the  number 
of  times  that  maneuver  was  demonstrated  or 
performed.  One  set  of  learning  curves  is 
presented  to  illustrate  the  progress  on  each 
maneuver  by  the  control  group  that  was  trained 
only  in  the  CH-47  aircraft  and  another  set  to 
show  the  progress  of  the  experimental  group 
that  trained  in  both  the  simulator  and  the 
aircraft.  Each  maneuver's  graph  has  three 
curves  on  it.  The  middle  of  the  three  curves 
represents  the  average  performance  of  the 
group  and  the  enveloping  curves  represent 
approximately  plus  and  minus  one  standard 
deviation.  The  plots  are  based  on  the  16th, 
50th,  and  84th  percentiles  of  the  groups. 
Therefore,  the  extreme  curves  encompass,  in 
both  performance  level  and  trials,  the  rea- 
sonable extremes  of  the  classes. 

OPERATIONAL  PROCEDURES 


Each  class  of  approximately  twelve  avia- 
tors reported  to  Fort  Rucker  several  days 
before  the  start  of  flight  training  for 
administrative  purposes  and  to  attend  ground 
school  in  preparation  for  the  flight  training. 

At  the  start  of  flight  training  both 
groups  spent  half  a day  together  in  ground 
school  and  split  the  second  half  day  for 
separate  flight  training  activities.  The 
control  group  underwent  flight  training  in 
the  CH-47  aircraft  according  to  the  program 
set  out  in  the  Flight  Training  Guide.  During 
this  training  the  IPs  used  the  daily  grade 
sheets  and  recorded  for  each  maneuver  the 
students'  performance  on  each  attempt,  per- 
formance on  the  subtasks,  and  the  time  spent 
on  the  maneuver.  The  IPs  were  instructed  to 
train  the  students  to  a performance  level  of 
six  on  the  twelve  point  scale.  This  level 
of  skill  was  considered  the  criterion  for 


passing  all  checkrides  and  an  acceptable  skill 
level  i''om  which  to  continue  training  after 
completing  the  course.  However,  training  was 
rarely  stopped  at  this  proficiency  level 
despite  instructions,  and  IPs  would  train  to 
levels  eight  to  ten  if  there  was  time  to  do  so. 
At  the  completion  of  advanced  training,  an 
aircraft  qualification  checkride  was  admin- 
istered and  upon  passing  it  the  aviator  be- 
came a qualified  CH-47  pilot. 

The  four  students  from  each  class  select- 
ed for  the  experimental  group  did  not  begin 
their  flight  training  in  the  CH-47  aircraft; 
they  began  training  in  the  CH47FS.  The  IPs 
training  them  in  the  simulator  followed  the 
same  Flight  Training  Guide,  taught  the  same 
maneuvers  in  the  same  order  and  to  the  same 
criterion  level  of  performance.  As  was  the 
case  with  the  control  group,  the  IPs  teaching 
the  experimental  group  recorded  performance 
level  and  practice  time  for  every  maneuver 
performed  on  the  daily  grade  sheets.  The  IPs 
were  instructed  to  train  all  basic  maneuvers 
to  skill  level  six,  the  criterion  level,  in  the 
simulator  or  until  it  was  obvious  that  a stu- 
dent would  not  reach  this  level  of  performance 
in  a reasonable  amount  of  time.  At  the  com- 
pletion of  training  the  basic  phase  maneuvers 
in  the  simulator,  the  students  were  given  the 
basic  phase  checkride  in  the  CH-47  aircraft. 
Unlike  the  control  group  requirement,  the 
checkride  did  not  have  to  be  passed.  Following 
the  basic  phase  checkride  in  the  aircraft  the 
experimental  group  returned  to  the  CH47FS  and 
continued  training  on  the  advanced  phase  maneu- 
vers. Upon  completion  of  this  phase  of 
training,  another  checkride,  similar  to  the 
aircraft  qualification  checkride,  was  given  in 
the  aircraft.  At  this  stage,  training  in  the 
simulator  was  stopped  and  training  began  in 
the  aircraft.  The  aircraft  training  was 
intended  to  teach  those  maneuvers  that  could 
not  be  taught  in  the  simulator  and  those  maneu- 
vers that  each  subject  had  not  passed  in  the 
last  aircraft  checkride.  As  will  be  discussed 
later,  considerably  more  training  was  usually 
given.  Training  in  the  CH-47  was  followed  by 
the  final  aircraft  qualification  checkride  in 
the  aircraft.  Performance  and  time  data  were 
also  collected  throughout  this  last  phase  of 
training. 


RESULTS  AND  DISCUSSION 


CUMULATIVE  TRANSFER  EFFECTIVNESS  RATIOS 


The  CTERs  for  each  maneuver  taught  in 
both  the  simulator  and  the  aircraft  appear  in 
Table  1.  The  table  presents  CTERs  calculated 
from  the  median  number  of  trials  spent 
training  in  the  simulator  and  the  median 
number  of  trials  to  proficiency  level  eight 
spent  training  in  the  aircraft. 
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TABLE  1 


CUMULATIVE  TRANSFER  EFFECTIVENESS 
RATIO  (CTER)  BY  MANEUVER  FROM  THE 
CH47FS  TO  THE  CH-47  AIRCRAFT 


Maneuver 

CTER 

Trials  to  Criterion 

General  Airwork 

1.00 

Cockpit  Runup 

1.50 

Four-Wheel  Taxi 

2.80 

Two-Wheel  Taxi 

1.00 

Takeoff  to  Hover 

.63 

Hovering  Flight 

.79 

Landing  from  Hover 

.69 

Normal  Takeoff 

.75 

Traffic  Pattern 

.61 

Deceleration 

1.25 

SAS  Off  Flight 

1 .33 

Normal  Approach 

.53 

Maximum  Takeoff 

1.25 

Steep  Approach 

1.00 

Shallow  Approach 

.58 

Confined  Area  Recon 

1.00 

Confined  Area  Approach 

.75 

Confined  Area  Takeoff 

.50 

External  Load  Briefing 

.67 

External  Load  Takeoff 

.50 

External  Load  Approach 

.50 

Pinnacle  Recon 

.50 

Pinnacle  Approach 

.00 

Pinnacle  Takeoff 

.33 

LEARNING  CURVES 


Learning  curves  of  maneuvers  that  were 
taught  in  both  the  simulator  and  the  air- 
craft are  presented  in  Figures  1 through  6. 

To  represent  continuity  of  training,  and 
learning  curves  of  the  experimental  group  in 
the  simulator  and  the  aircraft  are  presented 
on  the  same  graph.  For  ease  of  comparison, 
the  learning  curves  of  the  control  group  in 
the  aircraft  are  presented  directly  below 
the  experimental  group's  curves  for  each  maneu- 
ver. 

OVERTRAINING 


devices  and  that  transfers  perfectly  from  the 
simulator  to  the  aircraft.  The  CTER  snould 
be  1.0.  Assume  that  it  requires  15  trials 
to  learn  in  the  aircraft,  15  trials  to  learn 
in  the  simulator,  and  after  simulator  training 
requires  no  further  training  in  the  aircraft. 
Putting  these  figures  into  the  equation: 


CTER  = A/C  trials  cont  ^rp  - A/C  trials  exp  grp 
Simulator  trials  exp  grp 


CTER  = 


CTER  = 1 . 


As  expected,  the  CTER  indicates  that  the 
simulator  is  as  good  a trainer  as  is  the 
aircraft. 

Given,  for  example,  overtraining  of  5 
trials  in  both  devices: 


CTER  = 0,75 

The  resulting  CTER  gives  the  erroneous  impres- 
sion that  the  simulator  is  not  as  good  a 
trainer  as  is  the  aircraft.  This  is  typical 
of  the  CTERs  reported  in  Table  1.  The  usual 
case  was  to  overtrain  in  both  devices  as  can 
be  seen  from  the  learning  curves. 

The  CTERs  were  calculated  on  the  basis  of 
total  number  of  trials  in  the  simulator  since 
it  must  be  presumed  that  any  overtraining  here 
was  transferred  to  the  aircraft.  The  trials 
counted  in  the  aircraft  training  were  only 
those  needed  to  reach  criterion.  The  last 
example  recalculated  this  way: 


CTER  = 0.75 

This  again  gives  the  impression  that  the  simu- 
lator is  not  as  good  a training  device  as  is 
the  aircraft. 


One  of  the  factors  that  makes  the  inter- 
pretation of  the  CTERs  in  Table  1 difficult 
is  overtraining.  Overtraining  occurred  when 
trials  and  time  was  spent  in  training  an 
aviator  to  perform  a maneuver  at  a skill 
level  greater  than  level  six,  the  criterion 
performance  level.  The  following  examples 
will  illustrate  the  effects  overtraining  can 
have  on  CTERs . 

Consider  a hypothetical  maneuver  that  is 
properly  trained  to  a criterion  in  both 


In  an  earlier  report  (McGaugh  & Holman, 
1977)  these  CTER  data  were  presented  using  a 
criterion  performance  level  of  six,  the  level 
the  IPs  were  requested  to  use  in  training  as 
the  criterion  to  stop  training  on  a particular 
maneuver.  In  this  report  the  CTER  data  pre- 
sented in  Figure  1 are  calculated  on  the  basis 
of  a criterion  performance  level  of  eight. 

All  of  the  maneuvers  were  trained  to  a median 
level  of  at  least  eight  in  the  aircraft  and 
using  this  criterion  makes  the  CTERs  more 
typical  of  what  actually  occurs  in  training. 
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The  change  in  criterion  decreased  the  CTERs 
of  five  of  the  24  maneuvers  and  increased  the 
CTERs  of  13  of  the  24  maneuvers. 


COCKPIT  RUNUP 


The  following  maneuvers  were  selected  for 
discussion  as  typifying  the  various  patterns 
of  transfer  from  very  good  to  poor. 

GENERAL  AIRWORK 

General  Airwork  included  a number  of 
specific  maneuvers  such  as  climbs,  descents, 
and  turns.  This  skill  was  judged  only  once 
per  training  session  and  was  judged  almost 
every  session.  Consequently,  training  on 
this  maneuver  exceeded  criterion.  It  was 
usually  rated  while  flying  between  stagefields 
or  to  other  training  areas  in  either  the  simu- 
lator or  aircraft.  As  can  be  seen  by  the 
learning  curves  in  Figure  1,  General  Airwork 
was  overtrained  in  both  the  simulator  and  the 
aircraft.  The  overtraining  makes  it  difficult 
to  interpret  the  CTER  for  this  maneuver.  On 
a trials  to  criterion  basis,  the  CTER  is  1.0. 
Had  it  not  been  for  the  unavoidable  over- 
training, the  CTER  would  probably  have  been 
slightly  higher  than  1.0. 


TEST  GROUP 


Figure  1.  General  Airwork 


The  trials  to  criterion  CTER  is  1.5. 

This  CTER  and  the  learning  curves  (Figure  2) 
indicate  that  this  maneuver  was  learned  more 
efficiently  in  the  simulator  than  in  the  air- 
craft. It  is  often  the  case  that  when  the 
learning  task  is  procedural  in  nature  a 
training  device  provides  a more  effective 
learning  environment  than  that  provided  by 
the  actual  device. 


TEST  GROUP 


FOUR-WHEEL  TAXI 

The  learning  curves  for  Four-Wheel  Taxi 
(Figure  3)  show  that  this  maneuver  was  quickly 
learned  in  the  simulator  and  that  this  training 
transferred  well  to  the  aircraft.  The  CTER  is 
2.8.  Since  there  was  little  overtraining  in 
the  simulator  on  this  maneuver,  the  trials  to 
criterion  CTER  accurately  reflects  the  effec- 
tiveness of  the  trainer.  A CTER  greater  than 
1.0  indicates  that  Four-Wheel  Taxiing  is  better 
trained  in  the  simulator  than  in  the  aircraft. 
As  in  the  case  of  General  Airwork,  this  maneu- 
ver was  done  frequently  in  the  aircraft  as  a 
matter  of  necessity  even  after  training  could 
have  been  terminated. 
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TEST  GROUP 


TEST  GROUP 


r 


SIM  TRIALS  A/C  TRIALS 


CONTROL  GROUP 


Figure  3.  Four-Wheel  Taxi 
HOVERING  FLIGHT 

The  hovering  maneuvers  were  among  the 
most  difficult  to  perform  in  the  simulator. 
Training  on  them  transferred  to  the  aircraft 
with  less  efficiency  than  many  other  maneuvers. 

[ The  learning  curves  in  Figure  4 clearly  illus- 

trate this  point  and  are  typical  of  learning 
curves  generated  by  many  training  devices. 

The  simulator  group  required  more  trials  than 
the  aircraft  group  to  reach  median  criterion 
performance  level  six  (15  trials  vs  6)  in  their 
respective  devices.  When  starting  training  in 
the  aircraft,  the  performance  of  the  simulator 
group  dropped  well  below  its  rating  established 
at  the  end  of  simulator  training.  These  two 
effects  are  typical  of  the  transfer  of  training 
of  complex  tasks  from  a training  device  to  the 
actual  situation  and  are  indicated  by  CTERs  of 
less  than  1.0.  The  CTER  is  .79  and  is  somewhat 
higher  than  Takeoff  to  a Hover  and  Landing  from 
a Hover.  The  training  effectiveness  of  the 
simulator  for  these  three  hovering  maneuvers 
are  similar  and  all  indicate  that  the  simulator 
is  more  difficult  to  hover  than  the  aircraft. 

It  is  difficult  to  determine  why  this  is  the 
case,  but  it  is  oelieved  to  be  due  to  the 
limited  fi  nd  of  view,  the  infinity  focus 


Figure  4.  Hovering  Flight 


presentation  of  the  visual  system,  and  the 
lack  of  other  depth  cues. 

SHALLOW  APPROACH 

The  learning  curves  for  Shallow  Approach 
(Figure  5)  are  similar  to  those  for  the 
hovering  maneuvers.  The  Shallow  Approach 
requires  more  training  in  the  simulator  and 
transfers  to  the  aircraft  with  a decrement  in 
performance.  The  trials  to  criterion  CTER  are 
.58.  The  difference  in  training  transfer 
observed  between  the  Shallow  Approach  and  the 
Steep  Approach  (CTER  = 1.0)  is  attributed  to 
the  longer  time  spent  near  the  ground  in  the 
Shallow  Approach.  Near  the  ground  the  impor- 
tance of  the  visual  system  field  of  view  and 
focus  is  close  to  that  required  for  the 
hovering  maneuvers. 

PINNACLE  RECONNAISSANCE 

Along  with  the  Confined  Area  Reconnais- 
sance, IPs  believed  that  the  limited  visual 
field  of  the  simulator  eliminated  any  effec- 
tive training  of  Pinnacle  Reconnaissance. 
Therefore,  four  of  the  twenty-four  students 
in  the  simulator  group  were  not  trained  on 
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AIRCRAFT  TRIALS 
Figure  5.  Shallow  Approach 


AIRCRAFT  TRIALS 

Figure  6.  Pinnacle  Reconnaissance 


this  maneuver  in  the  simulator.  However,  the 
learning  curves  (Figure  6)  do  not  support 
this  conclusion  but  indicate  that  the  maneuver 
is  quickly  learned  in  both  the  simulator  and 
the  aircraft.  As  was  the  case  with  some  of 
the  other  maneuvers  (Confined  Area  Approach 
and  Landing)  which  were  learned  in  a very 
small  number  of  trials,  it  is  difficult  to 
determine  the  actual  training  effectiveness 
of  the  simulator.  The  CTER  is  .50  but  is 
based  on  such  a small  number  of  trials  that 
it  is  difficult  to  defend  with  confidence. 

The  learning  curves  for  Pinnacle  Approach  and 
Landing  are  similar  to  those  for  the  Pinnacle 
Reconnaissance  in  that  the  maneuver  is  learned 
quickly  in  both  simulator  and  aircraft  and  its 
CTER  is  zero. 

STUDY  II 

COMBAT  READINESS  FLYING 

METHOD 

The  objective  of  this  study  was  to 
experimentally  determine  the  extent  that 
combat  readiness  flying  skills  trained  in  the 


CH47FS  can  be  maintained  and  transfered  to 
the  aircraft.  The  determination  was  made  on 
a maneuver  by  maneuver  or  task  by  task  basis 
of  maneuvers  and  tasks  performed  both  in  the 
CH47FS  and  the  aircraft. 

The  method  used  followed  a pretest- train- 
posttest  paradigm.  The  subject  aviators  were 
given  an  extensive  flight  test  at  the  begin- 
ning of  the  program,  then  trained  in  the  CH-47 
FS  periodically  for  6- months  in  addition  to 
essential  aircraft  flying  and  then  were  given 
another  flight  test.  A second  similar  group 
was  given  the  two  tests  and  flew  the  aircraft 
but  did  not  receive  CH47FS  training. 

SUBJECTS 

The  subjects  for  this  study  were  32 
FORSCOM  aviators  qualified  in  and  currently 
flying  the  CH-47.  Sixteen  of  the  aviators 
comprised  the  experimental  group  that  received 
simulator  training  and  the  other  16  were  the 
control  group.  These  aviators  were  selected 
from  two  FORSCOM  posts  with  equal  numbers  of 
experimental  and  control  subjects  coming  from 
each  post. 
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INDEPENDENT  VARIABLE 

The  independent  variable  in  this  study 
was  the  use  of  the  CH47FS  in  a combat  readi- 
ness flying  (CRF)  training  program.  The  16 
control  aviators  were  requested  to  limit  thei>" 
flying  durinq  the  6-month  test  period  to 
mission  essential  flying.  Mission  essential 
flying  was  defined  as  flight  in  a CH-47  essen- 
tial to  the  support  mission  of  the  unit.  They 
were  specifically  requested  not  to  fly  for 
training  purposes  nor  to  fly  other  aircraft  or 
flight  simulators.  These  aviators  were  ex- 
empted from  meeting  required  flight  hours  and 
from  taking  required  flight  tests  for  the 
duration  of  the  study. 

The  16  experimental  aviators  were  treated 
the  same  as  the  control  aviators  except  for 
receiving  training  in  the  CH47FS.  Each  of 
these  test  aviators  received  30  hours  of 
training  in  the  CH47FS  during  the  6-month 
test  period.  The  training  was  given  in  5- 
hour  blocks  once  every  4 weeks  over  6 of 
the  4-week  cycles.  The  training  program 
was  designed  to  meet  F0RSC0M  needs  by  the 
F0RSC0M  training  representative,  and  included 
all  of  the  maneuvers  tested  (see  Table  2) 
plus  others  not  tested. 

PERFORMANCE  MEASURES 

The  performance  measures  essential  to 
this  study  were  the  ratings  given  each  maneu- 
ver performed  on  the  pre-  and  posttraining 
checkrides.  These  performance  ratings  were 
done  on  the  same  12-point  scale  used  in  Study 
I.  In  addition  to  checkride  ratings,  the 
experimental  group  was  rated  on  each  maneuver 
practiced  in  the  CH47FS  as  were  the  subjects 
of  Study  I . 

OPERATIONAL  PRCCEDURES 

The  pretest  inflight  checkrides  were 
administered  to  all  subjects  at  the  beginning 
of  the  6-month  study  period  by  their  unit 
standardization  pilots  under  the  supervision 
of  the  Test  Directors.  These  data  were  col- 
lected at  the  field  sites  and  brought  to  Fort 
Rucker  for  analysis.  At  this  point,  all 
subjects  began  flying  mission  essential 
flights  only  and  recorded  their  flight  expe- 
rience. 

Those  aviators  chosen  to  receive  training 
in  the  simulator  came  to  Fort  Rucker  at  4- 
week  intervals.  They  trained  in  the  CH47FS 
in  pairs,  two  pairs  per  week,  for  approximately 
6 months.  The  training  was  conducted  by  two 
FORSCOM  instructor  pilots  trained  to  operate 
and  teach  in  the  simulator. 

At  the  completion  of  training,  all  sub- 
jects, control  and  simulator  trained,  were 


given  a second  checkride  identical  to  the 
first.  Again,  these  checkrides  were  super- 
vised by  the  Test  Directors  and  the  data 
returned  to  Fort  Rucker  for  analysis. 

RESULTS  AND  DISCUSSION 

Data  presented  here  are  for  an  experi- 
mental group  of  15  and  a control  group  of  13 
aviators  rather  than  16  each.  Military  and 
personal  needs  required  four  of  the  subjects 
to  terminate  their  participation  before 
taking  the  posttraining  flight  test. 

Durinq  the  6-month  test  period,  the 
simulator  group  trained  in  the  CH47FS  for  a 
mean  time  of  29.7  hours  and  participated  as 
copilots  for  a mean  time  of  26.4  hours.  The 
mean  CH-47  aircraft  time  for  this  group  was 
45.2  hours  and  for  the  control  group  58.0 
hours.  This  difference  was  not  statistically 
significant  (t  (26)  = 0.27,  £>  .5). 

The  flight  test  scores  were  transformed 
by  a procedure  described  by  Hays  (1967)  to 
weight  the  score  for  each  maneuver  by  an 
estimate  of  its  difficulty.  The  difference 
between  the  control  group's  pre-  and  posttest 
score  (52.5  and  53.7)  is  not  significant 
(jt  (12)  = .98,  £ > .5).  The  difference 
between  the  simulator  group's  pre-  and  post- 
test  scores  (47.5  and  56.7)  is  significant 
(t  (14)  = 6.8,  £ < .002). 

Table  2 presents  the  mean  ratings  of  each 
maneuver  for  the  experimental  group.  The  pre- 
and  posttest  means  were  tested  for  significant 
differences  with  the  correlated  t-test  and  the 
£ value  and  its  significance  level  are  tabled. 
The  significance  levels  listed  are  for  the  two- 
tailed  _t-test  with  alpha  set  at  .05.  All  prob- 
abilities above  .05  are  considered  nonsignifi- 
cant. A similar  table  was  constructed  for  the 
control  group  and  only  two  of  the  35  maneuvers 
had  improved  significantly  (Straight  & Level 
Flight  and  2-Wheel  Taxi). 

OVERALL  TEST  SCORES 

The  significant  difference  between  the 
pre-  and  posttests  taken  by  the  simulator 
group  indicates  that  the  simulator  trained 
group  improved  its  performance  in  flight  in 
the  CH-47  as  a result  of  training  in  the 
CH47FS.  The  control  group  neither  improved 
nor  worsened  over  the  6-month  test  period. 

The  CH-47  mission  essential  flying  during 
the  study  period  maintained  this  group's 
flying  skill . 

MANEUVER  TEST  SCORES 

Table  2 compares  the  mean  pre-  and  post- 
test performance  by  the  experimental  group  on 
each  maneuver.  Seventy-four  percent  of  the 
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TABLE  2 


FLIGHT  TEST  SCORES  BY  MANEUVER  FOR  THE  SIMULATOR  GROUP 


I 


MANEUVER 

MEAN  TEST 

SCORES 

PRE 

POST 

t 

P< 

Cockpit  Runup 

6.6 

7.8 

3.1 

.01 

Taxi  (4-Wheel) 

6.6 

7.2 

2.6 

.02 

Takeoff  to  Hover 

6.6 

7.5 

2.0 

.05 

Hovering  Flight 

6.3 

7.3 

3.1 

.01 

Normal  Takeoff  from  Hover 

6.3 

7.7 

4.2 

.01 

Traffic  Pattern 

5.7 

7.0 

3.2 

.01 

Normal  Approach  to  Hover 

5.5 

6.7 

4.6 

.01 

Landing  from  Hover 

5.7 

7.2 

5.0 

.01 

Normal  Takeoff  from  Ground 

5.7 

7.1 

4.0 

.01 

Normal  Approach  to  Ground 

5.5 

6.9 

3.2 

.01 

Maximum  Takeoff 

5.4 

6.9 

3.7 

.01 

Steep  Approach 

5.5 

6.6 

2.1 

.05 

Standard  Autorotation 

5.1 

6.3 

1.6 

— 

Shallow  Approach  Single  Engine 

5.6 

7.5 

2.6 

.02 

Normal  Takeoff  w/NBC 

5.5 

6.3 

2.2 

.05 

Traffic  Pattern  w/NBC 

6.1 

7.4 

4.0 

.01 

Normal  Approach  w/NBC 

5.1 

6.3 

4.9 

.01 

Maximum  Takeoff  w/NBC 

5.5 

6.8 

3.3 

.01 

Standard  Autorotation  w/NBC 

5.2 

6.1 

1 .8 

— 

External  Load  Procedures 

Briefing  & Hook  Check  w/NBC 

5.7 

6.4 

1 .4 

— 

Takeoff  & Flight  w/NBC 

6.1 

6.5 

1 .2 

— 

Approach  & Landing  w/NBC 

5.5 

6.2 

2.0 

— 

Briefing  & Hook  Check 

5.8 

6.3 

1.2 

— 

Takeoff  & Flight 

6.3 

6.9 

2.0 

— 

Approach  & Landing 

5.5 

6.6 

3.9 

.01 

Instrument  Procedures 

Radio  Check 

7.7 

7.4 

0.6 

— 

Straight  & Level  Flight 

6.5 

7.3 

2.17 

.05 

Level  Turns 

6.2 

7.1 

2.4 

.05 

Straight  Climbs  & Descents 

6.1 

7.3 

3.5 

.01 

Approach,  GCA 

6.5 

6.7 

0.7 

— 

Taxi  (2- Wheel) 

5.4 

7.2 

4.7 

.01 

Cockpit  Shutdown 

7.0 

8.3 

4.4 

.01 

Emergency  Procedures 

Engine  Failure 

4.9 

7.3 

3.0 

.01 

Low  Side  Governor  Failure 

4.9 

7.1 

3.2 

.01 

Transmission  Oil  Low 

5.7 

6.6 

2.9 

.01 

tested  maneuvers  showed  a significant  improve- 
ment. The  largest  group  of  maneuvers  that 
showed  no  improvement  was  external  load  opera- 
tions. It  is  believed  that  this  was  due  to 
limitations  in  the  simulator's  visual  system. 
Autorotations  also  did  not  improve,  probably 
for  the  same  reason. 

The  results  of  Study  II  agree  with  those 
of  Study  I as  to  the  maneuvers  that  are  diffi- 
cult to  train  in  the  CH47FS.  Even  though  these 
maneuvers  were  difficult  to  train  in  the  simu- 
lator, there  is  no  evidence  of  negative 
training  on  any  maneuver  tested. 


CONCLUSIONS 

Study  I on  transition  training  and  Study 
II  on  combat  readiness  flying  both  show  that 
the  CH47FS  is  an  effective  training  device. 

For  many  maneuvers  Study  I indicates  that  it 
trains  as  well  or  better  than  the  CH-47  air- 
craft (CTERs  >_  1.0).  Another  group  of  maneu- 
vers can  be  trained  in  the  simulator  with  minor 
increases  in  the  amount  of  practice  required. 
These  maneuvers  had  CTERs  less  than  1.0  but 
greater  than  0.7.  A third  group  of  maneuvers 
had  CTERs  below  0.7.  However,  CTERs  in  the 
range  of  0.5  to  0.7  do  not  indicate  that  a 


training  device  is  ineffective.  Rather,  they 
indicate  that  the  training  device  is  not  as 
efficient  in  terms  of  the  number  of  trials 
required  to  learn  a particular  maneuver  as  is 
the  actual  aircraft.  In  the  case  of  a train- 
ing device  such  as  the  CH47FS  which  has  a 
large  proportion  of  maneuvers  with  consider- 
ably larger  CTERs,  it  also  leads  to  specula- 
tion about  the  characteristics  of  the  trainer 
that  results  in  these  lower  figures. 

The  maneuvers  that  produced  the  lower 
CTERs  were  all  maneuvers  in  which  a substan- 
tial part  of  the  maneuver  was  spent  close  to 
the  ground.  Examples  of  these  maneuvers  are: 
all  hovering  maneuvers,  shallow  approach, 
external  load  and  pinnacle  operations.  In 
Study  II  the  FORSCOM  aviators  did  not  increase 
in  proficiency  on  autorotations  and  external 
load  maneuvers.  It  is  believed  that  these 
effects  that  occur  close  to  the  ground  are  due 
to  limitations  in  the  visual  system. 

The  most  obvious  limitation  in  the  visual 
system  is  the  limited  field  of  view  (48°  x 
36°).  It  seems  that  many  of  the  visual  cues 
normally  used  for  hovering  maneuvers  are  more 
than  24°  from  the  centerline  of  the  aircraft 
or  180,  below  the  horizon.  These  cues  may  not 
be  essential  for  hovering  maneuvers,  as  is 
shown  by  the  fact  the  aviators  did  learn  to 
do  them,  but  are  available  in  the  aircraft 
and  do  aid  performance. 

A more  subtle  limitation  in  the  visual 
system  is  the  infinity  focus  CRT  display  in 
the  cockpit.  Through  the  use  of  a beam-split- 
ter and  a curved  mirror,  the  CRT  display  is 
made  to  cover  48°  visual  angle  and  is  focused 
at  infinity.  A near  object  seen  in  this 
system  does  not  have  all  the  usual  depth  cues 
present  to  indicate  how  close  the  object  is 
to  the  viewer.  Since  both  eyes  see  the  same 
scene  delivered  by  the  CRT,  there  is  no 
stereoscopic  disparity  present.  Since  the 
scene  is  focused  at  infinity,  there  is  not 
appropriate  angle  of  eye  convergence  or  lens 
accomnodation.  The  result  is  that  near 
objects  appear  to  be  farther  away  and  larger 
than  they  should  (Gregory,  1973).  This  dis- 
crepancy in  depth  perception  may  also  be 
partially  responsible  for  difficulties  in 
hovering  and  in  judging  distances  properly 
when  close  to  the  ground  at  the  end  of  a 
landing  approach. 

The  difficulties  in  judging  depth  in  the 
visual  display  are  especially  critical  in  the 


chin  window  display.  This  display  presents  a 
checkerboard  pattern  through  an  infinity  focus 
CRT  display  to  represent  the  ground  and  to 
provide  additional  cues  during  an  approach  or 
while  hovering.  Drift  cues  are  represented 
by  movement  of  the  pattern  across  the  display 
as  if  each  square  of  the  display  were  a 
square  on  the  ground  7-  feet  on  a side. 

The  only  depth  cue  available  in  the  display  is 
the  size  of  the  squares.  None  of  the  expected 
monocular  depth  cues  such  as  texture  gradients, 
overlapping  of  objects,  size  of  familiar 
objects  or  others  are  present.  As  in  the  main 
display,  there  are  no  stereoscopic  disparity, 
eye  convergency  or  lens  accommodation  cues. 

This  results  in  difficulties  in  judging 
height,  rate  of  closure,  and  rate  of  drift. 
These  cues  are  essential  to  performing  all 
maneuvers  close  to  the  ground  such  as  the 
landing  approaches  and  hovering  maneuvers 
found  to  be  difficult  to  perform  in  the 
CH47FS. 
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ENABLING  FEATURES  VERSUS  INSTRUCTIONAL  FEATURES 
IN  FLYING  TRAINING  SIMULATION 
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In  recent  years,  economic  and  resource 
constraints  have  forced  members  of  the 
training  community  to  actively  seek  more 
cost  - effective  approaches  to  routine 
training  needs  (Diehl  and  Ryan,  1977; 
Vandal,  1977;  McEnery  and  Lloyd,  1977; 
Provenmire,  Russel,  and  Schmidt,  1977). 
Within  the  Air  Force,  these  constraints 
have  resulted  in  efforts  to  reduce  the 
overall  number  of  flying  hours  (i.e.,  hours 
used  for  training  in  actual  aircraft)  by  25 
percent  by  the  early  1980 ' s (Flight 
Simulators,  1976).  In  order  to  accomplish 
this  goal,  the  Air  Force  is  moving  rapidly 
into  the  area  of  simulation  in  flying 
training  (Dunlap  and  Worthey,  1975).  While 
the  use  of  simulation  is  not  new  to  the  Air 
Force  (Valverde,  1968;  Smode,  1974;  Rivers 
and  VanArsdall,  1977),  use  of  simulation  on 
such  a broad  scale  as  that  directed  by 
Congress  is. 

Within  the  area  of  flying  training 
simulation,  the  concern  has  been  expressed 
(Caro,  1977ab)  over  the  extent  to  which 
instructional  methods  based  upon 
traditional  "in-flight"  models  provide  the 
most  effective  set  of  techniques  and 
procedures  for  conducting  training  in 
simulators.  Such  models,  while  obviously 
valid  for  teaching  persons  to  fly,  fail  to 
capitalize  upon  the  unique  capabilities  of 
simulators  to  free  the  instructional 
process  of  constraints  imposed  by  the  use 
of  operational  aircraft  as  training 
devices.  Inasmuch  as  in-flight 
instructional  models  promote  the  continued 
use  of  simulators  as  surrogate  aircraft, 
an  upper  limit  on  the  effectiveness  of 
simulators  is  set  by  the  limitations  of 
actual  aircraft  as  training  devices. 

Bridging  the  gap  between  continuation 
of  the  traditional  in-flight  model  of 
flying  training  instruction  and  a 
simulator-based  model  are  those  aspects  of 
a simulator  referred  to  as  "advanced 
training  features."  It  is  not  the  primary 


purpose  of  this  paper  to  review  the  present 
scope  of  advanced  training  features  avail- 
able for  the  conduct  of  flying  training. 
Descriptions  of  training  features 
believed  to  be  representative  of  those 
likely  to  be  found  on  early-  to  mid-1980 
generation  flight  simulators  have  been 
described  elsewhere  (Faconti,  Mortimer, 
and  Simpson,  1970;  Faconti,  and  Epps, 
1975;  Isley  and  Miller,  1976;  Knoop,  1973, 
Smith  and  Simpson,  1972).  Neither  is  it 
the  intent  to  propose  an  instructional 
model  for  the  most  effective  utilization  of 
such  features.  The  latter  effort  is 
currently  being  addressed  by  the  Air  Force 
Human  Resources  Laboratory  under  Project 
STRES  (Simulator  Training  Requirements  and 
Effectiveness  Study). 

Instead,  the  primary  purpose  is  to 
present  a conceptual  framework  for 
organizing  and  giving  direction  to  research 
and  development  in  the  area  of  advanced 
training  features.  . . a framework  that 

hopefully  will  not  only  bring  structure  to 
what  is  currently  a poorly  defined  area, 
but  that  will  also  promote  further 
instructional  research  into  utilizing  the 
"active"  instructional  capabilities  of  the 
modern  day  flight  simulator. 

Training  Features:  A Conceptual  Framework 

It  is  suggested  that  the  unique 
training  features  of  flight  simulators 
might  best  be  characterized  as  consisting 
of  (1)  enabling  features,  and  (2) 
instructional  features.  One  possible  scheme 
for  treating  the  differences  between  these 
two  types  of  features  is  given  below.  It 
is  hoped  that  the  framework  to  be  presented 
here  will  contribute  to  distinctions  among 
training  features  in  a manner  that  will 
also  aid  in  clarifying  those  dimensions 
along  which  the  effectiveness  and 
suitability  of  such  features  can  best  be 
evaluated.  While  the  chief  concern  here  is 
with  flight  simulators,  the  distinctions  to 
be  made  need  not  be  restricted  to  this  type 
of  simulation  device  alone. 
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I.  Enabling  Features.  Enabling 

features  arrange  for  the  occurrence  of 
physical  events  and  conditions  that  are 
necessary  to  support  training  but  not  for 
the  manipulation  of  these  events 
instructionally.  Their  training 

effectiveness  lies  in  their  ability  to 
create  the  conditions  under  which  training 
may  occur,  not  in  their  direct  effect  upon 
pilot  performance.  Enabling  features  are 
typically  the  "gi /en"  part  of  the  familiar 
three  part  behavioral  objective.  To  the 
extent  that  enabling  features  can  be 
separated  from  their  particular 
application,  the  relevant  dimensions  along 
which  their  effectiveness  should  be 
evaluated  are  fidelity,  ease  of  user 
operation,  domain  of  task  conditions 
simulated,  etc. 

Class  1:  Environmental  Conditions. 

Environmental  conditions  consist  of 
simulated  elements  of  the  natural  or 
man-made  environment  and/or  their  effects 
upon  the  aircraft  being  simulated.  (e.g., 
maneuver  and  disturbance  motion  cues; 
visual  sky/horizon/earth  scene;  sun  image; 
G seat/G  suit,  grayout/blackout,  target 
performance,  size,  and  display,  gaming 
area,  electronic  warfare  and  communications 
jamming,  runway  conditions, 

visibility/ceiling;  day/night;  other 
aircraft  as  in  refueling,  air-to-air  combat 
maneuvers,  or  formation  flight,  tactical 
conditions  and  targets,  cultural  features, 
moving  objects  such  as  truck  convoys, 
tanks,  boats  etc,  and  multiple  objects  such 
as  offensive  weapons,  SAMs,  antiaircraft 
artillery,  or  opposing  aircraft  launching  a 
missi le. 

Class  2:  Aircraft  Conditions.  Those 
features  which  relate  directly  to  the 
physical  operating  status  of  the  aircraft 
(e.g,  fuel  supply,  center  of  gravity, 
engine  status;  malfunctions,  etc.).  Such 
features  may  also  permit  manipulation  of 
the  performance  characteristics  of  opponent 
aircraft  (e.g.,  varying  the  percentage  of 
optimal  performance  of  the  opponent).  In 
instances  such  as  the  latter,  an  enabling 
feature  may  be  used  instructionally  for 
placing  the  student  at  a desired  advantage 
or  disadvantage. 

II.  Instructional Features. 

Instructional  features  consist  of  those 
provisions  (available  either  through 
software  manipulation  or  actual  hardware 
component)  by  which  the  operator  is  able  to 
manipulate  enabling  conditions  in  order  to 


bring  about  desired  changes  in  pilot  perfor- 
mance. Evaluations  of  the  effectiveness  of 
instructional  features  are  difficult,  if 
not  impossible,  to  make  independently  of 
the  manner  in  which  they  are  applied. 
Depending  upon  the  intended  function  of  the 
instructional  feature,  its  effectiveness 
may  be  measured  either  in  terms  of 
instructor/operator  performance  or  in  terms 
of  student  performance  directly.  It  is, 
however,  the  "effect"  produced  by  use  of  an 
instructional  feature  that  is  of  primary 
concern.  Since  instructional  features  do 
not  represent  "things"  in  any  real  sense, 
fidelity  is  not  a relevant  dimension  for 
their  evaluation. 

Class  1:  Passive  Instructional 
Features.  Those  instructional  features  for 
which  there  is  little  or  no  direct  contact 
with  the  student.  Passive  instructional 
features  would  include  CRT  and  graphic 
displays  used  by  the  instructor,  physical 
layout  and  actual  utilization  of  console 
hardware,  performance  measurement  in 
nonadaptive  systems,  procedural  monitoring 
capabilities,  etc.  In  general,  the  passive 
instructional  features  assist  the 
instructor  in  performing  monitoring  and 
evaluation  functions.  To  the  extent  that 
an  instructor  or  operator  must  interact 
with  information  presented  by  such 
features,  human  factors  criteria  are 
appropriate  both  as  a part  of  their  design 
and  their  evaluation.  To  the  extent  that 
some  designs  may  be  more  efficient  than 
others,  objective  criteria  based  upon 
actual  instructor  performance  in 
representative  training  settings  are 
required.  Evaluation  methodologies  such  as 
those  utilizing  benchmark 
instructor/operator  tasks  for  the 
evaluation  of  alternative  display  formats 
represent  one  alternative  for  further 
development  as  well  as  operational  test  and 
evaluation.  The  point  to  be  made  is  that 
while  such  features  are  used 
instructional ly,  the  primary  effect  is  one 
measured  in  terms  of  instructor  rather  than 
student  performance. 

Class  2:  Active Instructional 

Features.  Those  instructional  features  for 
which  there  is  direct  student  contact  with 
the  feature.  It  is  suggested  that  these 
features  may  be  further  subdivided  into  the 
following  classes: 

Subclass  A:  Those  features  which 
substitute  for  functions  provided  by  the 
instructor  in  real-time  and  which  may  only 


indirectly  contribute  to  more  efficient 
training  (e.g.,  recorded  preflight 
briefings  and  flight  demonstrations). 

Subclass  B:  Those  features  that 

contribute  to  more  efficient  training  by 
eliminating  or  reducing  "dead"  time  (e.g., 
use  of  freeze  and  preprogrammed  initial 
condition  sets). 

Subclass  C:  Those  features  which 
allow  the  instructor  to  augment  the 

physical  cues  available  to  the  student 
(e.g.,  visual,  auditory,  and/or  kinesthetic 
cues  not  normally  present  in  the  pilot's 
"natural  environment,"  auditory/visual 
performance  alerts,  etc),  or  to  use 

instructional  methods  not  available  in  the 
aircraft  (e.g.,  performance  record  and 
replay). 

Subclass  D:  Those  features  that 

enable  the  instructor  to  "restructure"  the 
basic  characteristics  of  the  task  or  the 
way  in  which  the  task  is  performed  (e.g., 
control  of  task  "tempo,"  ground  position 
freeze,  axis  (parameter)  lock,  etc.). 

The  Application  of  Instructional  Features 

While  elements  of  the  first  two  classes 
of  "active"  instructional  features 
contribute  to  more  efficient  flying 
training  (principally  through  making 
available  more  practice  time  per  session), 
neither  represents  a true  departure  from 
the  basic  elements  of  the  traditional 
in-flight  model  of  instruction.  The 
demonstration,  for  example,  is  most  often 
used  only  in  its  most  rudimentary  form, 
that  is,  as  a canned,  prerecorded  version 
of  an  in-flight  type  demonstration.  Rarely 
is  the  demonstration  capability  used 
jointly  with  other  features  to  create  an 
instructional  capabi 1 ity  beyond  that  which 
is  possible  under  normal  in-flight 
conditions.  While  significant  research 
problems  still  remain  in  these  areas  (for 
example,  determining  the  most  effective 
manner  in  which  to  manipulate  the  content 
and  placement  of  the  recorded 
demonstration),  innovative  applications  of 
simulation  to  flying  training  reside  in 
manipulation  of  Class  C and  Class  D type 
instructional  features. 

An  Example: 

Consider,  for  example,  the  following 
application  of  advanced  training  features 
to  the  training  of  an  air-to-surface 


weapons  delivery  task.  As  in  many  complex 
psychomotor  tasks,  the  ability  to  diagnose 
ones  own  errors  represents  one  of  the 
difficult  aspects  of  the  task.  In  the 

air-to-surface  task,  one  of  the  most 

difficult  aspects  to  convey  to  the  naive 
student  is  the  notion  of  "compensating 
errors."  While  the  "school  solution" 
involves  the  student's  being  able  to 
configure  the  plane  so  that  certain  release 
parameters  are  met,  more  often  times  than 
not  the  student  hits  the  target  because 
deviations  in  one  parameter  are  compensated 
for  by  deviations  in  a second  and/or  third 
parameter.  Under  present  methods  of 
instruction,  the  manner  in  which  these 
errors  compensate  is  learned  only  through 
repetition  of  the  task  either  in 
the  aircraft  or  in  the  simulator.  The 
conceptual  aspect  of  this  complex 
psychomotor  task  might  better  be  taught 
through  the  integrated  use  of  a number  of 
the  advanced  training  features. 

Consider  first  the  use  of  a prepro- 
grammed initial  condition  set  that  when 
executed  places  the  student  at  the  correct 
release  altitude,  dive  angle,  airspeed, 
etc.  Consider  now  the  additional  use  of  a 
bomb  impact  predictor  cue  (Hughes,  Paulsen, 
Brooks,  and  Jones,  1978;  Cyrus,  Templeton, 
and  McHugh  (in  press)  that  provides  a 
continous  and  immediate  depiction  on  the 
ground  of  where  the  bomb  will  impact. 
Finally,  consider  the  use  of  the  parameter 
freeze  or  axis  lock  feature  to  hold 
constant  the  student's  airspeed,  heading, 
and  altitude,  leaving  dive  angle  free  to 
vary.  By  employing  the  simulator  now  in  a 
ground  position  freeze  mode  and  giving  the 
student  control  over  the  stick  and  rudder, 
the  student  can  experiment  with  the  effects 
of  dive  angle,  for  example,  independently 
of  other  parameters  and,  thorough  use  of 
the  predictor,  see  the  results  of  these 
manipulations  immediately  on  the  ground 
without  the  normal  delay  associated  with 
the  flight  time  of  the  bomb.  By  freezing 
other  parameters  in  a similar  manner,  the 
student  is  able  to  see  directly  how 
corrections  in  one  parameter  are  able  to 
compensate  for  deviations  from  ideal  in 
another  parameter.  One  might  consider  using 
such  an  exercise  as  part  of  an  initial 
demonstration  in  addition  to  the 
traditional  demonstration  where  the  student 
sits  back  and  passively  watches  a "canned" 
performance  of  the  task. 

Once  the  student  begins  performing  the 
task,  the  instructor  might  consider  the  use 
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of  other  features.  For  example,  the 
feedback  delay  Inherent  In  the  bombing  task 
imparts  a significant  delay  between  the 
actions  of  the  pilot  at  the  release  point 
and  the  feedback  for  these  actions  obtained 
when  he  looks  back  to  observe  the  point  of 
impact  of  the  bomb.  Learning  theory  would 
suggest  that  such  a delay  degrades 
learning.  The  delay  might  be  eliminated  in 
one  of  two  ways.  In  cne  way,  the  system 
might  be  frozen  at  the  moment  the  pilot 
releases  the  bomb  and  the  impact  point 
immediately  illuminated.  While  the  freeze 
would  give  the  pilot  the  opportunity  to 
check  his  release  parameters  and  out  of 
cockpit  visual  references  without  having  to 
attend  to  flying  the  plane,  the  continuity 
of  the  performance  is  disrupted.  The  effect 
of  such  a disruption  on  the  acquisition  on 
a motor  task  is  not  known.  Another 
alternative  would  be  not  to  employ  the 
freeze,  but  to  illuminate  the  target  the 
moment  the  pilot  presses  the  trigger. 
Continuity  of  the  performance  is  thus  not 
disrupted  and  the  inherent  feedback  delay 
interval  is  eliminated.  As  with  any 
intervention  into  the  training  setting  that 
alters  real  world  conditions  for  the  sake 
of  training,  the  instructor  must  also 
consider  ways  in  which  to  systematically 
withdraw  such  cues. 

While  not  an  instructional  "feature" 
per  se,  the  principle  of  backward  chaining 
might  also  be  used  effectively  in 
performances  involving  the  chaining 
together  of  subtasks.  According  to  the 
principle  of  backward  chaining,  the 
terminal,  as  opposed  to  the  initial,  links 
of  the  chain  are  acquired  first.  On  the 
bombing  task,  for  example,  the  first  link 
of  the  chain  to  be  acquired  would  involve 
the  pilot’s  release  of  the  bomb  at  the 
correct  pickle  point.  The  system  might  be 
arranged  so  as  to  have  the  simulated 
aircraft  fly  this  segment  of  the  task  under 
computer  control  requiring  only  that  the 
student  press  the  bomb  release  button  at 
the  proper  time.  As  with  the  suggested 
applications  described  above,  the  freeze 
and  replay  capabilities  might  also  be 
intergrated  into  this  approach.  Once  the 
student  is  able  to  recognize  the  correct 
release  point,  that  portion  of  the  task 
between  release  and  the  time  the  pilot 
rolls  out  on  final  might  be  added  to  the 
chain.  As  performance  on  each  portion  of 
the  chain  reaches  criterion,  the  system 
would  arrange  for  the  next  portion  of  the 
chain  to  be  trained.  Similar  application  of 
such  a backward  chaining  approach  might 


also  prove  to  be  beneficial  in  tasks  such 
as  the  overhead  traffic  pattern, 
straight-in  approach  and  flair,  as  well  as 
in  such  tasks  as  carrier  landings  and 
aerial  refueling.  In  the  latter  task, 
training  might  begin  with  the  student 
attempting  to  maintain  contact  with  the 
tanker  boom,  proceed  next  with  making 
contact  from  a short  distance  out,  and  only 
then  proceed  to  making  the  initial  approach 
to  the  tanker  from  a normal  distance  out. 

A Further  Example 

Consider  a second,  perhaps  less 
complicated,  example  than  the  first  and  the 
type  of  research  design  that  might  be 
appropriate  for  evaluating  the 
effectiveness  of  alternative  instructional 
feature  applications.  The  particular 
example  involves  the  use  of  the  performance 
record/playback  feature  either  in  the 
recorded  demonstration  or  replay  modes  and 
weighs  the  benefits  of  using  the  feature 
against  those  to  be  derived  from  allowing 
the  student  to  continue  to  practice. 

During  the  course  of  acquiring  the 
skill  associated  with  performing  a 
particular  maneuver,  the  student  frequently 
watches  the  instructor  perform  a 
demonstration  of  the  task.  While  automation 
of  the  initial  demonstration  is  widely  used 
in  flying  training  simulation,  an  issue 
exists  as  to  the  relative  merits  associated 
with  (1)  repeating  the  original 
demonstration  (either  in  part  or  in  whole), 
(2)  making  available  to  the  student  or 
instructor  alternative,  prerecorded 
demonstrations  of  the  maneuver  for  viewing 
on  subsequent  trials,  or  (3)  forgoing  any 
repetition  of  a demonstration,  regardless 
of  format,  for  the  sake  of  allowing  the 
student  the  opportunity  for  further 
practice. 

Those  who  advocate  not  repeating  the 
original  demonstration  might  argue  that  the 
original  demonstration  serves  to  establish 
for  the  student  a "standard"  against  which 
he  or  she  uses  to  compare  his  or  her  own 
subsequent  performance.  To  repeat  the 
demonstration  a second  or  third  time  would 
simply  be  a duplication  of  the  function 
served  by  the  first  presentation.  It  might 
be  arugued  on  the  other  hand  that  the  naive 
student  does  not  on  the  first  viewing  of 
the  demonstration  attend  to  all  the  proper 
elements  of  the  demonstration,  and  only 
with  repeated  viewings  grasps  the  full 
intent  of  the  demonstration.  A less 
obvious,  but  equally  likely  possibility,  is 
that  repeated  demonstrations  serve  to  break 
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up  periods  of  massed  practice,  giving  rise 
inadvertently  to  an  intermittent  practice 
effect. 

There  are  those  who  argue  for  the 
effectiveness  of  repeated  demonstrations, 
but  who  call  attention  to  the  need  for 
adapting  such  demonstrations  to  the 
particular  needs  of  that  student  at  that 
particular  moment.  These  persons  further 
argue  that  no  prerecorded  set  of 
demonstrations  will  be  found  to  be  ideally 
suited  to  such  individualized  use.  For  such 
persons,  a practical  alternative  to  the 
use  of  the  recorded  demonstration  might  be 
the  playback  feature.  By  recording  the 
performance  of  the  student  on  each  trial 
for  subsequent  playback,  the  instructor  has 
the  option  for  having  the  student  view  a 
performance  that  presents  the  very  errors 
the  instructor  wants  to  draw  attention  to 
(in  fact,  the  student's  very  own  errors), 
to  overlay  on  this  visual  presentation  a 
narration  that  is  ideally  suited  to  that 
particular  student  at  that  particular  point 
in  time,  and  to  present  whatever  portion  of 
that  previous  performance  he  desires. 

The  alternatives,  all  realistic  and 
feasible,  give  rise  to  an  experiment  where 
the  following  experimental  conditions  are 
present. 

Condition  1:  Following  an  initial 
demonstration  of  a maneuver,  the  student 
practices  for  some  x-number  of  trials,  sees 
a second  presentation  of  the  original 
demonstration,  practices  x-trials,  etc. 
until  some  predetermined  number  of  trials 
have  been  completed. 

Condition  2:  Following  an  initial 
demonstration  of  a manevuer,  the  student 
practices  for  some  x-number  of  trials, 
rests  for  a period  of  time  equal  to  the 
duration  of  the  demonstration  viewed  by 
subjects  in  Condition  1,  practices  for  some 
x-trials,  rests,  practices,  etc.  until  some 
predetermined  number  of  trials  have  been 
completed. 

Condition  3:  Following  an  initial 
demonstration  of  a maneuver,  the  student 
practices  for  some  x-number  of  trials, 
continues  to  practice  during  that  period  of 
time  alloted  to  students  in  Condition  1 for 
subsequent  viewings  of  the  demonstration, 
pratices  for  x-trials,  etc.  until  some 
predetermined  number  of  trials  have  been 
completed. 


Condition  4:  Following  an  initial 
demonstration  of  a maneuver,  the  student 
practices  for  some  x-number  of  trials,  then 
views  an  instructor-narrated  playback  of 
performance  on  his  last  trial,  continues  to 
practice,  views  playback,  etc,  until  some 
predetermined  number  of  trials  have  been 
completed. 

Condition  5:  Following  an  initial 
demonstration  of  a maneuver,  the  student 
practices  for  some  x-number  of  trials, 
views  a live  instructor  demonstration, 
practices  for  x-trials,  views  live 
demonstration,  practices  etc.  until  some 
predetermined  number  of  trials  have  been 
completed. 

While  the  spacing  of  demonstrations, 
playbacks,  or  whatever  is  a potentially 
important  variable,  their  placement  in  the 
suggested  study  is  arbitrary.  So  too  is  the 
student's  "need"  for  the  instructional 
event  at  the  time  it  is  programmed  to  occur 
in  the  study.  The  study,  nevertheless,  would 
serve  to  evaluate  alternative  instructional 
uses  of  the  advanced  features  in  terms  of 
conditions  having  pragmatic  consequences. 
Furthermore,  it  addresses  the  potential 
situation  where  the  time  -consuming  use  of 
an  instructional  event  such  as  a 
demonstration  or  playback  may  be  less 
preferred  than  continued  practice  on  the 
part  of  the  student. 

The  Need  for  Continued  Research 

While  technological  advances  are  likely 
to  lead  to  the  more  efficient  design  of 
instructor/operator  consoles  and  to  more 
efficient  management  of  the  simulator's 
training  features,  basic  instructional 
research  is  needed  to  determine  the  most 
effective  manner  for  utilizing  these 
features  in  actual  training  settings.  As 
the  move  toward  more  fully  automated 
training  continues,  one  may  expect  to 
observe  more  and  more  functions  currently 
performed  by  the  instructor  to  come  under 
computer  control  and  management.  There 
exists  a limit,  however,  as  to  what  extent 
such  an  engineering  oriented  approach  to 
automation  will  ever  fully  utilize  the 
unique  capabilities  of  the  modern  day 
flight  simulator.  It  is  anticipated  that 
the  eventual  outcome  of  such  an  approach 
will  simply  be  the  automation  of  an 
instructional  approach  derived  from  an 
in-flight  model  of  flying  training 
instruction.  The  insufficiency  of  such  an 
approach  for  simulation-based  flying 
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training  instruction  has  already  been 
alluded  to  here  and  elsewhere. 

In  flying  training  simulation,  as  in 
any  other  area  of  training,  engineering 
principles  simply  cannot  be  looked  to  as 
providing  the  basis  for  a technology  of 
instruction.  The  application  of  engineering 
principles  must  be  looked  to  as  providing  a 
means  to  an  end.  . . an  end  that  is  defined 
behavioral ly.  Such  a behavioral  definition 
of  flying  and  the  instructional  process 
that  supports  flying  training  represents  a 
key  effort  in  current  research  of  the 
Flying  Training  Division  of  the  Air  Force 
Human  Resources  Laboratory. 
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difficulties  to  overcome  in  the  evaluation  of  The  fidelity  and  realism  of  the  device  were 

aircraft  training  systems  effectiveness.  The  high;  and  complete  visual,  motion  and  aural 

main  difficulty  stems  from  the  complexity  of  cues  were  supplied  to  the  gunner.  The  device 

the  issue,  which  results  in  a lack  of  clear  also  provided  immediate  feedback  to  the  train- 

operational  definitions.  The  Naval  Training  ee  as  well  as  objective  performance  measurement. 

Equipment  Center  has  initiated  a research 

program  in  Training  Effectiveness  Evaluation  Contemporary  training  issues  revolve 

(TEE),  an  initial  part  of  which  is  to  develop  around  the  levels  of  fidelity  required  for 

a model  of  the  TEE  process.  The  main  objec-  simulation  and  the  costs  associated  with  such 

tive  of  the  initial  phase  of  the  program  is  fidelity.  In  ground-based  aircraft  simulation, 

to  define  the  elements  needed  in  TEE  and  to  for  example,  the  requirement  for  motion  cues 

specify  the  interrelationships  among  critical  has  not  been  addressed  satisfactorily.  This 

elements.  Another  objective  is  to  foster  bet-  issue  is  of  great  economic  interest  since 

ter  communications  within  the  training  and  platform  motion  bases  are  expensive  and  have 

operational  community.  a great  impact  on  engineering  design  and  facil- 

ity requirements.  Other  contemporary  issues 
INTRODUCTION  that  require  attention  are  illustrated  in  the 

technical  development  of  a 360°  visual  display 
One  early  attempt  at  aircrew  training  was  and/or  advanced  means  to  substitute  color  in 

a device  known  as  "pin  ball."  The  objective  a monochromatic  presentation.  A crucial  con- 

of  "pin  ball"  was  to  train  "live  fire"  sortees  sideration  is  the  question  of  how  much  more 

against  other  aircraft  (Air  Force,  1945).  effective  a training  system  could  be  with  the 

This  was  accomplished  by  gunners  firing  addition  of  such  technical  achievements, 

special  "frangible"  bullets  which  would  not 

damage  the  armored  target  aircraft.  The  tar-  The  effectiveness  of  a training  system 

' get  aircraft  had  110  microphones  mounted  on  directly  affects  the  performance  of  the  output 

the  fuselage.  Any  "hit"  was  registered  (i.e.,  the  graduates)  in  post-training  environ- 

through  a microphone  connected  to  a counter  ments.  Thus,  the  evaluation  of  training  effec- 

located  in  the  cockpit.  A hit  also  activated  tiveness  is  a critical  link  in  the  design  and 

a red  light  in  the  nose  of  the  target  aircraft 


use  of  simulators  and  training  devices.  It  Is 
clear  that  training  effectiveness  has  a direct 
Impact  on  combat  readiness. 

Contemporary  TEE  Difficulties 

One  of  the  major  difficulties  In  the  area 
of  TEE  Is  the  lack  of  agreement  on  definitions 
for  appropriate  terms.  For  example,  Johnson 
(1976)  noted  that  no  one  can  provide  a precise 
definition  of  training  effectiveness.  Other 
investigators  have  questioned  the  basic  valid- 
ity of  simulator  training  programs.  "The 
effectiveness  of  almost  all  military  simulator 
programs  is  being  assumed,  and,  in  many  cases, 
these  assumptions  may  be  In  error  by  signifi- 
cant amounts.  There  has  been  virtually  a total 
absence  of  controlled  tests  designed  to  vali- 
date military  simulator  training  programs" 
(Caro,  1977,  p.ll). 

TEE  at  NAVTRAEQUIPCEN 

The  Naval  Training  Equipment  Center  (NAV- 
TRAEQUIPCEN) is  Involved  in  research  programs, 
as  well  as  applied  efforts  to  understand  both 
the  methodological  and  practical  considera- 
tions underlying  TEE.  The  program  has  taken 
on  increased  significance  due  to: 

( Advances  In  the  state  of  the  art  in 
simuation  technology. 

„ Heavy  emphasis  on  determining  the  costs 
related  to  simulation. 

Requirements  to  train  personnel  to  per- 

• form  increasingly  complex  aircrew  tasks 

and  tactics. 

Elements  of  TEE 

The  objective  of  TEE  In  aircrew  training 
is  to  assess  how  well  a training  system  imparts 
skills  and  knowledge  to  a trainee,  and  how  .veil 
the  skills  and  knowledge  are  demonstrated  in 
real  world  task  performances.  The  process  de- 
pends upon  three  interdependent  elements: 

• The  trainee. 

• The  training  environment. 

• The  operational  environment. 

Training  system  development  has  concentrated 
on  the  optimum  Interaction  between  the  trainee 
and  training  environment;  TEE  concentrates  on 
the  interaction  between  the  training  and  opera- 
tional environments. 

A vital  requirement  in  training  effective- 
ness evaluation  is  that  there  must  be  a match 
between  what  is  required  in  the  operational 
environment  and  what  is  trained.  The  match 
requires  a clear  specification  of  operational 
requirements,  and  the  development  of  pertinent 


training  criteria  and  related  performance 
measures.  The  development  of  criteria  must 
occur  through  a systematic  process  which  ex- 
amines longitudinal  effects  to  ensure  that  all 
relevant  operational  factors  are  Included  In 
a TEE.  Goldstein  (1974)  stated  the  Issue 
succinctly.  "Evaluation  Is  only  one  part  of 
a long-term  systematic  approach;  therefore.  It 
Is  necessary  to  pay  particular  attention  to 
developing  relevant  criteria  of  learning  and 
transfer  performance"  (Goldstein,  1974,  p. 21 3) . 

Another  important  requirement  in  TEE  is 
the  need  for  effective  two-way  communication 
and  feedback  between  the  operational  and  train- 
ing communities.  Unfortunately,  such  communi- 
cation links  appear  to  work  ineffectively  in 
many  cases.  Without  such  links,  training  will 
tend  to  be  divorced  from  operational  input?. 
Further  emphasis  must  be  placed  on  developing 
more  adequate  connunications. 

An  effective  TEE  requires  that  considera- 
tion must  be  given  to  the  methods  by  which 
TEE's  are  conducted.  TEE  often  has  been  con- 
ducted through  the  transfer  of  training  (TOT) 
paradigm.  TOT  consists  of  comparing  training 
performance  to  performance  which  can  be  ob- 
served and  measured  in  an  operational  setting 
(Blaiwes,  et.  al.,  1973).  The  comparison  is 
difficult,  as  many  divergent  operational 
requirements  must  be  integrated  to  develop 
criteria.  Another  specific  method  used  to 
measure  training  effectiveness  is  the  incre- 
mental transfer  effectiveness  function  (ITEF) 
which  deals  with  the  time  savings  in  a train- 
ing situation  based  on  the  utilization  of 
successive  increments  of  training  in  another 
usually  less  costly,  training  situation 
(Povenmire  and  Roscoe,  1973).  This  is  a 
useful  measure  to  express  time  savings,  but, 

It,  like  many  other  measures  does  not  inte- 
grate training  costs  into  its  formulation. 

Tne  usual  assumption  is  that  it  is  less  costly 
to  use  a simulator  for  training.  Yet  Caro 
(1977)  stated  that  no  one  knows  which  simula- 
tor programs  are  cost-effective  since  it  is 
difficult  to  determine  all  the  variables  that 
affect  costs. 

Other  considerations  for  a TEE  include 
the  attitudes  of  personnel  (trainers,  trainees, 
operational  personnel)  and  the  impact  other 
variables  in  a total  training  system  have  on 
TEE.  The  variety  of  considerations  make 
training  effectiveness  evaluation  a complex 
effort.  A useful  way  to  view  the  many  vari- 
ables and  their  interactions  Is  through  the 
development  of  a multidimensional  TEE  model. 

A Model  of  the  TEE  Process 

A model  of  the  TEE  process  is  being 
developed  to  provide  guidance  for  the  deter- 
mination of  how  well  a training  system  meets 
operational  requirements.  The  preliminary 
TEE  Model  (TEEM)  is  depicted  in  Figure  2. 
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FIGURE  2 - TRAINING  EFFECTIVENESS  EVALUATION  MODEL  (TEEM) 


The  TEE  Model  was  designed  to  examine  the 
connection  between  operational  requirements 
and  training  system  components  as  shown  in  the 
central  portion  of  Figure  2.  For  example,  an 
operational  task  such  as  a take-off  must  be 
broken  down  into  knowledge  objectives  and 
skill  objectives  which  serve  as  the  basis  for 
the  evaluation.  The  breakdown  procedure  is 
accomplished  by  means  of  the  Instructional 
Systems  Development  (ISD)  process.  Through 
this  process,  necessary  task  analyses  of  job, 
duty,  task  and  task  elements  are  performed  to 
specify  training  requirements. 

The  model  also  provides  guidelines  to 
develop  factors  that  are  not  in  the  central 
portion,  but  are  very  important  considerations 
in  a TEE  nonetheless.  For  Instance,  the  model 
considers  the  Influence  from  diverse  subjec- 
tive factors  including  personnel  perceptions, 
attitudes  and  views;  or  objective  factors  such 
as  influences  from  unit  procedures.  Factors 
such  as  future  trainee  duty  assignments  and 
experiences  (longitudinal  factors)  must  enter 
into  the  definition  of  operational  influences. 

Influences  on  the  central  portion  of  the 
model  also  occur  from  training  factors  (left 
hand  area  of  the  model).  Resources,  training 
personnel,  facilities,  devices  and  trainee 
ability  all  enter  into  training  evaluation 
from  the  trainee  and  training  environment. 
While  most  of  the  Influences  are  objective, 
subjective  attitudes  of  the  trainee  and 
training  personnel  must  be  factored  into  a 
TEE. 

The  major  benefits  of  the  model  are  that 
it  can  serve  as  a conriunl cation  tool  between 
operational  and  training  groups,  and  that  it 
provides  an  analytical  tool  for  examining  the 
TEE  process  in  detail.  Both  of  these  aspects 
of  the  model  are  vital  to  the  Increased  uti- 
lization of  simulator  outputs  which  Include 
trained  personnel  as  well  as  advances  In  sim- 
ulation state  of  the  art.  The  model  is  a 
necessary  first  step  to  define  issues  and 
problems  related  to  TEE  and  to  guide  future 
research  efforts. 

The  model  may  also  be  used  to  develop  a 
checklist  of  the  elements  for  a TEE  Including 
the  following: 

a.  From  the  operational  environment 
element  — 

# Have  the  operational  requirements 
been  fully  identified? 

t Have  personnel  perceptions  of  the 
requirements  been  analysed? 

Have  actual  and  future  squadron 

* practices  been  considered? 


b.  From  the  training  environment  ele- 
ment — 

Have  training  device  and  simula- 

• tor  costs  been  properly  iden- 
tified? 

, Have  training  personnel  attitudes 
been  determined. 

c.  From  the  trainee  element  — 

• What  are  the  trainees'  views  of 
the  training  program? 

d.  and  from  the  model  — 

• Have  valid  performance  criteria 
been  developed? 

Have  all  relevant  costs  influenc- 
ing the  training  been  identified? 

The  model  also  can  serve  as  a means  of 
integrating  on-going  research  results  and 
attempts  at  developing  performance  measure- 
ments. One  Air  Force  effort  will  attempt  to 
apply  automated  performance  measurement  tech- 
niques within  a C-5A  aircraft.  The  objective 
of  this  effort  will  be  to  compile  operational 
data  which  can  be  compared  directly  to  a sim- 
ulated system.  Another  effort,  which  is 
reminiscent  (in  objectives)  of  the  "pin  ball" 
system  described  earlier,  is  the  Fairchild 
TOW  Helicopter  Installed  Television  Monitor 
and  Recorder  (HITMORE)  developed  for  real- 
time monitoring  and  assessment  of  aerial  gun- 
nery performance.  This  system  provides  imme- 
diate post-mission  playback  and  analysis  of 
gunner  aim-point  during  live  or  simulated 
firings  of  the  AN-IS-TOW  COBRA  Weapon  System 
(Ayral  and  Chandler,  1978). 

Additional  Research  in  TEE 

The  model  as  developed  requires  valida- 
tion and  improvement,  but  it  can  form  a basis 
for  defining  TEE  problems  and  solutions.  In 
almost  all  endeavors  it  is  important  to  define 
elements  of  a problem  before  attempting  to 
specify  solutions.  This  is  especially  true 
with  TEE  since  it  is  a complex  topic.  The 
methodologies  to  address  many  of  the  elements 
have  not  been  completely  specified  and  need 
additional  research.  Some  methodologies  that 
might  be  beneficial  to  TEE  are: 

Multidimensional  scaling  for  perfor- 

• mance  measurement  may  offer  a prom- 
ising way  to  assess  crew  proficiency. 

The  measurement  of  longitudinal 
variables  such  as  retention  must  be 

# addressed;  transfer  of  training  to 
the  operational  environment  must 
show  retention,  otherwise  the  train- 
ing program  may  not  be  worth  the  cost 
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Cost-effectiveness  analysis  has  not 
e been  successfully  applied  to  TEE; 
relevant  costs  must  be  determined. 

With  the  Increasing  role  that  TEE  now 
plays  and  will  play  In  the  future  of  Naval  air- 
crew training,  a comprehensive  research  plan 
must  be  developed  to  address  Issues  such  as 
these.  TEE  must,  therefore,  be  Included  as  an 
Integral  part  of  all  training  systems. 
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INTRODUCTION 

Training  Devices,  in  one  form  or  another, 
have  been  with  us  ever  since  man  first  discovered 
that  by  conveying  his  experienced  skills  to  those 
who  were  inexperienced,  he  could  more  readily 
house,  feed,  and  defend  himself.  The  requirement 
to  maintain  training  devices  has  also  always  been 
with  us. 

Training  devices  took  a giant  step  forward 
with  Ed  Link's  invention  of  the  famous  Link 
Trainer.  Since  that  time,  each  step  in  the 
development  of  training  devices  has  been  tied  to 
growth  during  periods  of  war  or  peacetime  crises. 
With  each  step,  however,  the  problems  associated 
with  training  devices  became  more  severe.  The 
cost  of  software  has  more  than  doubled;  the  cost 
of  hardware  has  grown  greatly;  and  personnel 
related  costs  have  steadily  risen,  from  46%  of 
the  Defense  budget  in  1968  to  61%  in  1976.* 
Dramatic  increases  in  the  cost  of  training,  logistic 
support,  retirement  benefits,  and  administration 
of  the  military  and  Civil  Service  work  force  have 
plagued  the  Department  of  Defense. 

Contract  maintenance  has  been  viewed  by 
many  as  the  logical  answer  to  the  dilemma  of 
doing  more  with  less  dollars.  This  paper  will 
discuss  many  of  the  pros  and  cons  of  contract 
maintenance  to  support  the  contention  that 
although  it  might  not  be  THE  ANSWER  in  each 
situation,  contract  maintenance  is  indeed  a viable 
alternative  that  should  be  carefully  considered 
when  weighing  all  maintenance  factors. 

BACKGROUND 

The  advent  of  the  Link  Trainer  was 
followed  by  World  War  II  and  the  expanded  use 
and  explication  of  training  devices.  Although  the 
cost  of  trainers  was  relatively  inexpensive  when 
compared  to  the  cost  of  aircraft,  there  were 
many  new  problems  to  overcome.  The  resolution 
of  these  problems  led  to  new  concepts  in  training 
evaluation  and  measurement,  procurement, 
maintenance,  and  logistic  support,  and  new 
assessments  as  to  the  qualifications  and  training 
of  trainer  maintenance  personnel. 

Due  in  port  to  the  chill  of  the  Cold  War, 
the  communist  scare  of  the  late  1940s  and  early 
1950s,  the  growing  military  might  of  Communist 
China,  and  the  lessons  of  the  Korean  War,  the 
need  for  a highly  trained  fighting  force  did  not 
diminish.  In  those  days,  as  many  of  you  may 
recall,  a nation's  military  might  was  measured  by 


the  number  and  sophistication  of  its  various 
weapon  systems.  Material  needs  took  precedence 
over  humanistic  needs  as  " people " took  a backseat 
to  the  steady  procurement  and  development  of 
new  weaponry. 

As  our  weaponry  became  more  complex,  so 
did  the  skills  necessary  to  maintain  it.  The 
military  countered  with  a broad  system  of 
personnel  screening  and  classification,  technical 
training,  on-the-job  training,  and  improved 
management,  control,  reporting,  procurement, 
maintenance,  end  logistic  support.  The  name 
"synthetic  trainers"  was  applied  to  virtually  all 
training  devices. 

Synthetic  trainers  were  basically  simple 
electromechanical  devices  that  tried  to  duplicate 
certain  aircraft  functions.  Our  flight  crews  were 
well  into  the  Jet  Age  while  synthetic  trainers 
were  still  trying  to  improve  the  performance  and 
capability  of  long  outdated  systems.  Many 
decision  makers,  both  in  Congress  and  the 
military,  questioned  the  validity  of  trainers  as  an 
effective  aircrew  training  aid.  In  addition,  large 
numbers  of  aircrews  resented  the  mechanical 
monsters  and  saw  them  as  a threat  to  actual 
flying. 

Between  1950  and  the  early  1970s,  there 
were  budget,  personnel,  and  program  cuts.  These 
cuts  were  influenced  by  the  race  for  space  and 
a shift  of  priorities,  the  Cuban  and  Lebanon 
Crises,  the  Arab-lsraeli  Wars,  major  changes  in 
our  defense  posture,  new  concepts  and  philosophy, 
Vietnam,  a wholesale  shift  in  the  world  economy, 
and  the  rebirth  and  rise  of  the  Arab  States. 

On  the  sidelines  and  virtually  unnoticed, 
the  synthetic  trainers  slowly  matured  and  were 
replaced  by  the  far  more  complex  simulator,  but 
the  entire  training  devices  field  remained  hidden 
in  shadow.  While  the  sophistication  of  aircraft 
systems  and  other  weaponry  created  a high  degree 
of  specialilzation  and  a vast  new  array  of  military 
job  skills,  the  training  devices  field  remained 
relatively  small  and  the  technician  was  then,  as 
now,  a jack-of-all-trades.  Personnel  assignment 
and  classification  policies,  technical  training,  and 
maintenance  were  basically  the  same  in  1972  as 
they  were  in  1952.  With  the  development  of  the 
versatile  computer,  a whole  new  technology  was 
being  applied  to  simulators. 


Despite  early  warning  signs,  the  Energy 
Crisis  of  1973/74  caught  the  entire  nation 
unprepared.  Almost  immediately  there  were  new 


notional  problems  and  a new  set  of  values  — and 
the  dramatic  theme  was  energy  conservation. 

The  Federal  Government  swung  into  action. 
The  GAO,  OMB,  the  President's  Scientific 
Advisory  Board,  as  well  as  earlier  McNeff  Report 
keyed  on  the  increased  use  of  training  devices 
to  reduce  flying  training  costs  and  to  conserve 
energy.  The  military  services  were  directed  to 
institute  a comprehensive  analysis  of  their  present 
and  future  simulator  requirements.  The  Air  Force 
alone  requested  more  than  150  new  simulators 
and  major  modifications  to  existing  devices.  The 
cost  of  this  simulator  modernization2and  research 
was  tagged  at  three  billion  dollars. 


The  use  and  technology  of  simulators 
advanced  at  on  unprecedented  pace  — and 
simulators  moved  from  the  shadows  into  the  stark 
reality  of  the  Computer  Age.  Thus,  in  1974, 
time  and  progress  finally  caught  up  with  training 
devices  and,  within  the  military,  the  small,  well 
hidden  training  devices  field  was  totally 
unprepared  for  the  technological  invasion. 


This  invasion  has  resulted  in  rising  costs 
in  all  areas,  major  problems  in  simulator 
maintenance,  and  problems  in  logistic  support. 
Most  of  the  major  problems,  however,  have  been 
personnel  related. 


Despite  the  extraordinary  efforts  since  the 
inauguration  of  the  all-volunteer  armed  forces, 
recruiting  has  barely  achieved  its  objectives.  In 
addition,  the  number  of  noncombatant  support 
personnel  has  grown  at  an  alarming  rate  and  many 
personnel  with  high  school  diplomas  simply  lack 
the  basic  skills  to  master  today's  technological 
demands.  These  demands  are  also  compounded 
by  the  exodus  of  experienced  enlisted  personnel 
who  are  highly  trained  in  the  skills  that  are  most 
salable  to  industry.  The  long  lead  times,  the 
lengthy  and  expensive  training,  the  technological 
explosion,  and  the  shrinking  defense  dollar  have 
mandated  that  DOD  examine  alternatives  for 
system  support.  These  alternatives  have  included 
the  increased  use  of  Civil  Service  and  contract 
maintenance  for  the  traditional  enlisted 
manpower,  or  a combination  of  these  alternatives. 
Since  they  do  not  "go  to  war,"  simulators  are 
particularly  viable  candidates  for  contract 
maintenance. 

The  simulator  industry  has  numerous 
examples  of  ongoing  contractor  maintenance 
programs:  Singer  and  American  Airlines  are 

supporting  the  Undergraduate  Pilot  Trainers  (UPT); 
AAI  maintains  both  the  Undergraduate  Navigator 
Trainer  (T45  UNT)  and  the  Simulator  (for) 
Electronic  Warfare  Training  (T5  SEWT);  Singer 
supports  advanced  simulators  at  both  Williams 
AFB  and  Luke  AFB;  and  Northrup  has  the  entire 
base  support  function  at  Vance. 


While  there  are  more  and  more 
considerations  for  contract  maintenance  by 
various  DOD  entities,  the  "bottom  line”  is  the 
cost  comparison  between  organic,  or  in-house,  and 
contract  maintenance.  There  are,  however,  pros 
and  cons  to  contract  maintenance. 

PROS  AND  CONS 

What  are  the  advantages  of  contract 
maintenance  and  why? 

First  and  foremost,  personnel  costs  are 
lower  for  contract  maintenance  than  organic 
and/or  Civil  Service  support.  This  is  a definite 
concern  since  personnel  costs  now  account  for 
approximately  65%  of  the  DOD  budget  and  the 
emphasis  is  on  reduced  manning  of  noncombatants 
(or,  if  you  will,  increasing  the  "teeth-to-tail" 
ratio). 

One  major  cost  of  personnel  is  that  of 
training.  The  cost  of  a 24-  to  36-week  technical 
school  course  ranges  from  $20,000  to  $25,000. 
This  does  not  include  advanced  skill  level  training, 
Type  I training,  special  training,  and  various  OJT, 
management,  social  actions,  NCO  leadership,  and 
advanced  leadership  courses.  The  cost  of  training 
Civil  Service  personnel  is  somewhat  higher  due 
to  the  higher  rates  in  pay  and  per  diem. 

While  in  training,  the  individual  is 
essentially  nonproductive.  Because  of  production 
delays  or  other  unforeseen  circumstances,  the 
military  technician  or  Civil  Service  employee 
could  find  himself  trained  and  ready  — but  with 
no  equipment  to  work  on.  Although  the  individual 
may  be  employed  elsewhere,  a loss  of  technical 
proficiency  on  the  device  for  which  the  individual 
was  initially  trained  will  probably  occur. 

Inherent  in  many  training  devices  is  the 
requirement  for  an  operator  and,  in  some  cases, 
an  operator/instructor.  With  most  training 
devices,  an  operator  can  be  easily  trained  within 
a month.  The  instructor  usually  requires  two  to 
three  months  to  reach  the  desired  proficiency. 
Because  of  the  classification  system,  ALL  training 
devices  personnel  must  attend  the  technical 
school.  Of  the  24-  to  36-week  school,  only  2 
weeks  are  directed  toward  operator  duties  and 
skills  while  the  remaining  22  to  34  weeks  are 
geared  toward  maintenance  knowledge  and  skills. 
Despite  the  $20,000  to  $25,000  technical  school 
investment,  many  school  graduates  remain 
operators  throughout  their  initial  enlistment. 
Additionally,  personnel  find  that  they  must  prove 
themselves  as  operators  before  undergoing  OJT 
to  make  them  knowledgeable  and  productive 
maintenance  personnel.  In  many  cases,  fully  three 
years  (of  a four-year  enlistment)  have  passed 
before  the  first  term  enlisted  technician  becomes 
an  integral  part  of  the  on-equipment  maintenance 
team.  In  effect,  a near-$25,000  training 
investment  is  not  commensurate  with  the 
maintenance  output  of  one  productive  year! 


Under  contract  maintenance  programs,  the 
contractor  is  responsible  for  and,  more 
importantly,  liable  for  providing  fully  trained, 
capable  maintenance  personnel  at  the  point  he 
accepts  responsibility  and  for  the  life  of  his 
contract. 

The  contractor  normally  achieves  lower 
costs  by  providing  fewer  people  to  perform 
equivalent  jobs.  Unimpeded  by  supervisory 
classifications,  nonjob  related  training, 
meetings/briefings,  manning,  and  other 
requirements,  the  wise  contractor  seeks  to 
maximize  personnel  efficiency.  With  personnel 
that  are  normally  more  experienced  and  of  higher 
skill  levels  than  military  personnel,  the  contractor 
has  no  manning  constraints  and  can  usually  keep 
his  supervisory  staff  at  a minimum. 

In  most  cases,  the  contractor  can  supply 
an  experienced,  stable  work  force.  If  training  is 
required,  the  contractor  sends  fewer  personnel 
and  the  training  is  tailored  to  exact  needs.  The 
contractor  can  also  achieve  economy  by  having 
his  training  school  graduates  train  the  remaining 
work  force.  Due  to  the  experience  levels  of  his 
people,  contractor  training  can  usually  be 
compressed  into  much  less  time.  Finally,  because 
of  his  flexibility,  the  contractor  does  not  require 
a long  lead  time  for  training.  His  training 
program  can  be  readily  geared  to  the  production 
or  modifications  schedule  of  the  device  in 
question.  As  a result,  the  loss  of  technical 
proficiency  is  unlikely. 

A significant  advantage  of  contractor 
maintenance  is  assignment  stability.  The  axiom 
that  "maintenance  is  maintenance"  is  akin  to 
exclaiming  that  if  one  can  fly  a Piper  Cub,  one 
should  be  equally  proficient  at  flying  a Boeing 
7^7.  Each  simulator  and  each  simulator  system 
has  its  own  peculiarities.  The  theory  that 
separates  pilot  from  navigator,  ECM  from 
gunnery,  model  board  from  dome  visual,  and  a 
six-degree  from  a four-degree  motion  base  plays 
a primary  role  in  the  maintenance  of  that 
equipment.  There  is  no  known  substitute  for 
experience.  In  many  cases,  about  the  time  the 
enlisted  technicitn  feels  at  home  on  the  device, 
he  is  reassigned.  Although  the  personnel 
assignment  system  can  withhold  assignment  under 
certain  conditions,  it  rarely  happens.  The 
contractor  does  not  have  this  problem. 

An  item  of  cost  in  personnel-related  issues 
is  that  of  administration.  The  administrative 
time  and  costs  associated  with  the  military  are 
significant.  The  contractor  is  bound  by  contract 
to  manage  and  control  his  employees;  thus  he 
essentially  performs  all  the  functions  formerly 
accomplished  by  the  various  military  supervisory 
and  command  levels.  In  some  cases,  contractor 
maintenance  provides  <n  appreciable  reduction  in 
the  personnel  administration  areas. 


Another  major  consideration  is  risk.  We 
know  of  no  system  or  procedure  whereby  the 
Government  is  relieved  of  all  risk.  Even  if  a 
contractor  fails  to  perform  and  incurs  a monetary 
liability  for  his  failure,  the  Government  is  still 
left  with  an  unfilled  program  or  training 
requirement.  We  must  also  recognize  that  the 
contractor  is  in  the  business  to  show  profit.  He 
lays  his  reputation  on  the  line  with  each  contract 
and  most  contractors  would  rather  pump  in 
whatever  is  required  than  to  admit  failure.  If 
good  news  travels  fast,  bad  news  travels  even 
faster.  Thus,  we  believe  there  is  a degree  of 
shared  risk  by  both  the  contractor  and  the 
Government. 

As  the  buyer,  the  Government  sets 
conditions  and  establishes  performance  criteria  for 
contractor  maintenance.  In  essence,  the 
contractor  who  bids  on  the  proposal  is  stating 
that  he  can  meet  the  requirements,  is  willing  to 
assume  the  monetary  risks,  and  can  do  the  job 
at  less  cost  than  a Government  work  force.  In 
a training  device  utilization  program  that  is  highly 
structured  and  tied  to  a rigid  incoming  and 
outgoing  student  pipeline,  an  efficient  contractor 
can  greatly  reduce  the  risk  and  possible  damage 
to  the  course  and  student  pipeline. 

A final  factor  for  contract  maintenance  is 
a potentially  lower  development  cost.  This  can 
occur  when  the  contractor  is  permitted  to  develop 
only  that  data  which  he  himself  deems  necessary. 
For  example,  a contractor  could  feasibly  maintain 
a complex  system  using  only  schematics,  wire 
lists,  interconnecting  diagrams,  and  engineering 
sketches.  Obviously,  this  major  cost  reduction  in 
development  denies  the  Government  the  capability 
to  solicit  future  competitive  bids  and/or  assume 
the  organic  maintenance  responsibility  without 
incurring  considerable  cost. 

While  on  the  subject  of  simulator 
development,  one  cannot  escape  the  enormous 
cost  of  a modern  simulator  system.  Major 
modifications  have  also  compounded  simulator 
costs.  This  cost  represents  a "sunk"  cost  and 
the  only  sure  method  to  recoup  the  investment 
is  to  maximize  the  use  and  capability  of  the 
simulator  system  throughout  its  life  expectancy. 
This  equates  to  optimum  maintenance  efficiency. 
Moreover,  the  new  breed  of  state-of-the-art 
devices  has  taxed  the  military's  ability  to  keep 
pace.  Technical  innovation  not  only  affects 
personnel  but  logistics  support  as  well.  Here, 
too,  contract  maintenance  can  fill  in  the  gap. 
Many  of  the  factors  cited  above  are  not  new. 
Military  decision  makers  have  been,  and  are, 
dealing  with  the  perplexing  problems  associated 
with  simulator  maintenance.  The  natural 
alternative  to  a military  work  force  is  Civil 
Service;  however,  with  even  higher  pay  rates  and 
administrative  costs,  Civil  Service  has  many  of 
the  same  constraints  as  the  military. 
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The  industry  has  cited  numerous,  cost 
effective  contract  maintenance  operations.  In 
one  instance,  nearly  75  military  and  Civil  Service 
employees  were  replaced  by  26  contractor 
personnel. 


The  prime  disadvantage  of  contract 
maintenance  is  the  untenable  position  of  the 
Government  if,  for  any  reason,  the  contractor 
fails  to  perform.  This  situation  can  be  especially 
acute  if  the  Government,  as  mentioned  earlier, 
has  not  procured  the  necessary  engineering  and 
technical  data.  It  would  be  difficult,  if  not 
impossible,  for  another  contractor  to  assume  the 
risk.  Should  the  Government  elect  to  take  over 
the  maintenance  responsibility,  personnel  would 
have  to  be  located  and  trained  and,  in  the 
interim,  there  would  be  limited  productivity  of 
the  device  and  the  people  associated  with  it.  In 
any  case,  the  cost  to  the  Government,  in  time, 
performance,  and  dollars,  would  be  significant. 


Another  argument  against  contract 
maintenance  deals  with  the  sole  source 
contractor.  The  contractor  may  have  developed 
the  technical  data  or  be  highly  specialized  to  the 
point  where  there  is  no  competition  or  he  may 
have  simply  maintained  the  device  for  so  long 
that  no  one  else  is  interested  in  competing  for 
the  contract.  In  short,  the  contractor  becomes 
the  recognized  expert  and,  as  such,  what  is  to 
prevent  him  from  escalating  the  cost  beyond 
reason?  Along  this  same  line,  there  is  yet 
another  argument  against  contract  maintenance. 
Stated  simply,  the  contractor  is  in  business  to 
make  a buck;  thus,  he  may  take  shortcuts,  run 
a "body  shop,"  and  only  perform  just  to  get  by. 

These  are  certainly  strong  arguments; 
however,  they  are  not  absolute.  Our  industry  is 
built  on  free  enterprise  and  free  competition. 
Just  as  the  Government  watches  industry,  we 
watch  each  other.  We  are  indeed  in  business  to 
make  a profit  and  we  can  obtain  that  profit  and 
continued  profit,  only  by  satisfying  the  customer 
or  buyer.  If  we  fail,  we  have  not  only  tarnished 
our  reputation  throughout  the  industry,  but  have 
negated  our  chances  for  other  Government 
contracts  as  well.  Moreover,  the  Government  is 
the  buyer  and  it  is  the  Government,  not  the 
contractor,  who  establishes  the  conditions  and 
performance  levels  of  the  contract.  The 
Government  can  and  should  minimize  its  risk  by 
making  the  training  of  a replacement  a provision 
of  the  contract.  This  would  serve  to  protect 
both  the  Government  and  program  continuity. 


Some  of  the  major  pitfalls  of  contract 
maintenance  were  also  discussed  in  an  Air  Forc^ 
Air  Command  and  Staff  College  research  paper. 
The  author  skillfully  debated  the  issues,  negative 
aspects,  and  remedies  for  the  Government  to 
avoid  many  of  the  problems  related  above. 


Also  considered  disadvantages  of  contract 
maintenance  are  the  real  and  perceived  problems 
of  Contractor/Government  interface  and  the 
attendant  turmoil  inherent  in  this  situation. 
Frankly,  there  is  usually  some  difficulty  in 
integrating  the  contractor  into  a military 
environment.  Despite  a growing  number  of 
contract  operations,  there  are  a vast  number  of 
military  personnel  who  neither  know  nor 
understand  the  meaning  of  a non-personal  services 
contract.  Conversely,  many  contractor  personnel 
do  not  know  military  practices  and  procedures. 
The  tendency  of  the  military  is  to  place  everyone 
in  the  organizational  chain  and  to  exercise  some 
degree  of  control.  The  tendency  of  the 
contractor  is  to  resist  this.  The  resultant 
situation  strains  relationships,  affects  morale,  and 
hinders  operations.  The  contractor  "works  for" 
and  is  responsible  to  his  contracting  officer. 
Legally,  in  most  cases,  all  Government 
communications  pass  through  the  contracting 
officer  to  the  contractor  and  vise  versa.  This 
procedure  is  sometimes  misunderstood  and  often 
disagreed  with  by  the  military  operations  and 
maintenance  hierarchies.  In  practice,  however, 
both  parties  must  recognize  that  rapid  and  direct 
communications  are  advantageous  to  both.  The 
importance  of  amiable  relationships  and  rapport 
cannot  be  overemphasized  for  mutual 
understanding  is  the  primary  basis  for  good 
contractor  relations. 

To  minimize  the  dangers  of  strained 
relationships  and  misunderstandings,  all  parties 
should  be  made  aware  of  the  organizational 
structure,  performance  specifications,  and  the 
responsibilities  - and  nonresponsibilities  - of  the 
contract  maintenance  team.  Where  minor  out  of 
scope  requirements  are  required  and  have  a minor 
iripact  on  the  contractor,  we're  quite  confident 
the  customer  would  receive  the  requested  service 
without  added  cost.  In  our  business,  too,  the 
name  of  the  game  is  to  keep  the  customer 
satisfied. 

Large  out  of  scope  requests  that  require 
a major  expenditure  in  time  or  manhours  will 
certainly  result  in  contractor  payment  requests. 
Normally,  the  contractor  does  not  expect 
remuneration  for  added  effort  of  his  own  making; 
however,  he  will  usually  seek  payment  when  the 
effort  is  not  his  fault.  This  brings  up  another 
aspect  of  contractor  maintenance:  weekend, 

holiday,  special,  or  unforeseen  training  or  work 
and  the  threat  of  contractor  strikes  or  work 
slowdowns/stoppages.  The  most  obvious  solution 
is  to  build  contingencies  for  these  situations  into 
the  contract.  As  for  the  threat  of  union 
squabbles  and  contractor  strikes,  cursory 
investigation  indicates  that  the  Government  has 
experienced  far  more  difficulties  with  its  own 
Government  employees'  unions  than  with  simulator 
industry  contract  maintenance  operations.  (This, 
however,  is  not  necessarily  true  with  reference 
to  simulator  contractor  production  facilities.)  In 
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addition.  Government  union  agreements  and 
regulations  also  restrict  the  flexibility  of  Civil 
Service  employees  to  respond  to  shift  changes, 
weekend/holi  day  work,  and  training  schedule 
changes.  It  should  be  noted  that  all  Civil  Service 
overtime  must  be  approved  before  it  can  be 
authorized.  Most  contractors  do  not  have  this 
constraint  on  their  work  force.  By  building 
performance  measures  (as  opposed  to  time  and 
material  requirements)  into  the  contract,  the 
contractor  must  respond  and  the  Government  could 
care  less  how  he  schedules  his  employees  to  meet 
the  requirement. 

Another  complaint  regarding  contractor 
maintenance  is  the  question  of  manning  in  o 
national  emergency,  the  manning  of  overseas  sites 
or  operations  in  combat  zones.  Various 
contractors  have  maintained  successful  overseas 
operations  for  years  - and  that  also  includes 
Korea  and  Vietnam  during  wartime.  Of  course, 
simulators  do  not  go  to  war  and  for  the  most 
part,  their  high  cost,  fragile  equipment,  and  need 
for  air  conditioning  do  not  lend  themselves  to 
combat  operations.  As  for  the  high  cost 
associated  with  overseas  contract  operations,  we 
know  of  no  existing  low-cost  operation  overseas 
or  in  a combat  area.  Where  operating  costs  are 
high  and  the  risk  is  great,  the  contract  costs 
will  be  commensurate  - and  that's  the  way  it  is. 

A final  argument  against  contract 
maintenance  is  the  determination  of  performance 
standards  or  measurements.  As  stated  earlier, 
all  participating  parties  should  be  aware  of  and 
fully  understand  the  contract  maintenance 
measures  of  acceptable  performance.  The 
question  is  "what  should  that  performance  be?" 
or  "how  do  you  evaluate  the  performance  of  a 
contractor?"  The  same  rules  that  apply  to  the 
military  or  Civil  Service  do  not  necessarily  apply 
to  the  contractor.  As  an  example,  most 
Government  and  military  regulations  and  manuals 
are  designed  to  control  the  output  through  a high 
degree  of  supervision  and  control  over  the  work 
force.  The  legal  ramifications  of  a nonpersonal 
services  contract  are  such  that  the  Government 
is  expressly  forbidden  to  supervise  or  control 
contractor  employees.  This  does  not  mean  the 
Government  relinquishes  total  control  over  the 
output;  it  merely  means  the  output  should  be 
contractually  defined. 

The  usual  performance  criterion  has  been 
students  in,  or  scheduled,  versus  students  out,  or 
completed.  While  this  is  one  valid  measure,  it 
is  by  no  means  the  only  measure.  Other  tangibles 
such  as  logistic  support  cost,  the  extent  of  local 
repairs  - or  the  lack  thereof,  top  priority 
requisitions,  and  user  comments  can  be  effectively 
utilized  to  measure  contractor  performance  more 
accurately. 


There  may  not  be  an  answer  to  the  question 
of  what  contractor  maintenance  performance 
measures  should  be  used,  but  this  is  an  area  for 
further  investigation  by  both  Government  and 
industry. 


CONCLUSIONS 


We  have  discussed  the  history,  the  pros, 
and  the  cons  of  contractor  maintenance.  But 
what  does  it  all  mean?  Is  contractor  maintenance 
the  answer  — or  alternative? 

We  believe  that  the  pros  of  contractor 
maintenance  far  outweigh  the  cons  and  while 
contractor  maintenance  may  not  be  the  answer 
for  all  simulators,  it  definitely  should  be 
considered  as  a viable  support  alternative. 

To  be  objective,  it  is  extremely  difficult 
to  avoid  bias  on  any  subject  wherein  one  has  a 
vested  interest.  To  be  sure,  the  major 
considerations  for  simulator  contractor 
maintenance  are  less  cost  and  a high-degree  of 
flexibility.  The  arguments  against  were  quickly 
dispatched,  at  least  on  paper.  In  practice, 
however,  disagreements  are  not  always  easily 
resolved.  In  our  business,  it  is  virtually  impossible 
to  hide  failure,  incompetence,  or  unsound 
practices.  The  Government  obtains  some  degree 
of  protection  by  contracting  with  reputable, 
proven,  and  reliable  firms,  for  they  have  the  most 
to  lose  should  something  go  awry.  This  success 
or  failure  of  any  contract  is  not  dependent  upon 
the  actions  of  the  home  office  but  upon  the 
individuals  or  teams  in  the  field.  With  this  in 
mind,  a close  rapport  and  a mutual  understanding 
are  two  of  the  primary  keys  to  successful 
contractor  maintenance. 

We  believe  all  would  agree  that  contractor 
versus  organic  support  merits  further 
investigation.  Cost  avoidance  can  be  realized 
from  the  standpoints  of  labor  savings  and,  if  the 
Government  desires,  Engineering  and  Technical 
Data  costs.  The  Government  does  run  the  risk 
of  being  in  a sole  source  situation  with  a 
contractor,  particularly  if  engineering  technical 
data  is  not  procured,  and  it  is  true  that  in  the 
past,  some  major  contractors  have  gone  out  of 
the  training  device  business  which  could  leave 
the  Government  in  an  untenable  position. 

Again,  we  believe  the  pros  far  outweigh 
the  cons  and  recommend  that  the  Government 
expand  its  consideration  of  candidates  for 
contractor  maintenance  as  well  as  investigate 
areas  impacting  contractor  maintenance  activities, 
such  as  contractual  vehicles  and  methodologies 
for  measurement  of  performance. 
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Why  contractor  maintenance?  To  sum  up 
all  the  rhetoric  in  two  words:  less  cost. 
President  Carter  summed  it  up  neatly  when  he 
stated:  "When  the  Government  must  perform  a 
function,  it  should  do  it  efficiently.  Whenever 
free  competition  would  do  a better  job  of  serving 
the  public,  the  Government  should  stay  out"” 
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SIMULATION  OF  THUNDERSTORMS 


MELVIN  PAIKEN 

Grumman  Aerospace  Corporation 
Beth  page.  New  York 


INTRODUCTION 


Thunderstorm  presence  is  required  to 
fully  complement  a weather  environment  simu- 
lation. An  in-flight  aircraft  will  be  sub- 
jected to  forces  in  part  created  by  moving 
air  in  this  environment  which  can  be 
described  by  clear  air  turbulence  character- 
istics. 

When  the  aircraft  enters  a thunderstorm 
area,  the  forces  acting  on  the  aircraft  due 
to  moving  air  can  undergo  a large  change. 


There  are  other  phenomena  related  to 
thunderstorms  that  also  have  to  be  consider- 
ed : 

o Lightning 
o Thunder 
o Icing 
o Radio  Noise 

o Intensity  Level  Reflectivity 
TYPICAL  STORM  CELL  DEFINITION 


Reference  is  made  to  Figure  1 that 
illustrates  an  isometric  view  of  a typical 
simulated  storm  cell.  The  cell  is  defined 
as  a parallelepiped  that  encompasses  an 
inner  volume  containing  the  simulated  storm. 
The  vertical  distance,  h , defines  the 
altitude  of  the  lowest  face  of  the  cell.  The 
vertical  length  of  the  cell  is  defined  by 
h . Cell  coordinates  (X.,,  Y..,  h..)  define 
the  location  of  any  point  internal  to  the 
cell.  The  corner  boundaries  of  the  cell  are 
referenced  to  a prescribed  Gaming  Area  in 
terms  of  earth  coordinates  and  altitude 
(X,  a,  h ).  The  volume  below  the  storm  plane 
containsswinds  consisting  of  either  rising, 
falling  or  horizontal  drafts.  A library  of 
storms  can  thus  be  made  available  described 
by  intensity  aF,  or  ap,  and  an  associated 
height,  h . 


Reference  is  made  to  Figure  2 that 
illustrates  a planar  view  of  a typical 
storm  cell  model.  The  storm  is 


contained  within  a predetermined  area  bounded 
by  a *2  mile  square  (by  definition).  The 
shaded  area  represents  the  storm  environment 
and  the  remaining  area  within  the  square  de- 
picts clear  air  turbulence. 

CHARACTERISTICS 


Thunderstorms  in  general  can  occur 
individually  over  relatively  small  areas  or 
they  may  exist  along  a wide  front.  The  indi- 
vidual storms  can  be  described  as  a "cell" 
and  the  frontal  area  storms  can  be  described 
as  a "storm  line." 

Throughout  most  geographic  areas, 
thunderstorms  are  continuously  forming  into 
violent  turbulent  wind  states  and  decomposing 
with  dissipating  winds  accompanied  by  intense 
rainfall . 

During  the  initial,  or  cumulous,  phase 
of  a thunderstorm,  storm  intensity  increases 
due  to  the  presence  of  an  unstable,  moist  air 
mass  that  is  carried  upward  by  strong  drafts. 
This  moist  air  takes  on  the  form  of  globules 
of  water  which  in  turn  forms  into  snow  and 
ice  pellets  as  the  air  stream  works  further 
upward. 

The  initial  storm  phase  is  identified  by 
random  updrafts  whose  velocity  can  vary  be- 
tween 10  and  50  miles  per  hour  at  the  storm 
base  during  periods  between  two  and  six 
seconds.  These  intensities  increase  as  the 
updrafts  rise  in  altitude  within  the  thunder- 
cloud, or  storm  cell.  During  this  phase,  air 
is  continuously  being  drawn  inward  and  upward 
from  below  the  storm  base. 

As  the  storm  increases  in  size,  the 
characteristics  of  the  wind  currents  change. 
This  second,  or  mature,  phase  is  described 
as  follows: 

At  some  point,  the  density  of  water 
globules  and  ice  pellets  dannot  be  sustained 
by  the  upward  air  thrust  and  downward  wind 
currents  start  to  form.  The  proportion  of 
downward  air  currents  continues  to  increase 
accompanied  by  rainfall  and  possible  hail  or 
ice  particles.  The  storm  clouds  continue  to 
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grow  and  tend  to  scatter  out  at  the  top.  In 
this  case,  the  downward  winds  increase  in 
intensity  as  they  descend.  During  this  phase, 
the  winds  below  the  cloud  base  are  both  flow- 
ing outward  and  inward  from  the  storm  cell. 

The  final,  or  dissipating  phase  of  the 
storm  consists  of  intense  rainfall  with  down- 
ward winds.  The  rainfall  and  winds  gradually 
slacken  and  the  storm  is  dissipated. 

STORM  DYNAMICS 

The  cumulous  storm  phase  consists  of 
wind  intensities  (hs-hN)+K2)  where: 

N1  is  a Gaussian  random  positive  amplitude 
N2  is  a Gaussian  random  time  period 
is  a proportionality  constant 
hs  is  the  storm  base  altitude 
hN  is  an  altitude  within  the  storm  limits 
K2  is  the  storm  constant  intensity  at  hs 

The  wind  intensity  below  the  storm  is 
described  as: 

aRB  = NlVK35  + K 'Z> 

where : 

K3*K4  are  constants 

u is  the  unit  vector  in  the  storm  base  plane 
z is  the  unit  vertical  vector 

Falling  vertical  winds  are  represented  by 
intensities  given  by: 

“F  = -NlN2Kl(hc  + hs  - hN>  + K2}  1 
where: 

hc  is  the  storm  altitude  limit  above  h$ 

STORM  LINES 

Thunderstorms  can  occur  over  a broad 
front.  As  a consequence,  a storm  front  can  be 
represented  by  groups  of  cells  described 
above  within  realistic  presentation  areas  of 
between  10  and  200  square  miles.  A catalogue 
of  typical  storm  cells  S,,  S2,  S^.-.-S  is  to 
be  provided  for  insertion  inztheJGaming  Area. 
The  location  is  to  be  preselected.  Reference 
is  made  to  Figure  3 which  illustrates  a repre- 
sentative multiple  intensity  storm  front. 


A thunderstorm  area  may  also  consist  of 
a combination  of  multiple  storm  lines  and  in- 
dividual cells.  Reference  is  made  to  Figure4 
that  illustrates  a typical  thunderstorm  Gam- 
ing Area.  Storm  lines  are  precatalogued  and 
located  as  shown  by  Sl^.Slo,  S\.o  etc. 

Storm  front  S.  , is  shown  locatea’at  Gaming 
Area  coordinates  (^l3 , 8 |_,,  h|,).  Individual 
storms  S,,  S2>  Sj,  etc.  are  likewise  located 
by  Gaming  Area  coordinates. 

An  aircraft  flight  path  as  shown  in 
Figure  4 may  experience  a wind  intensity  pro- 
file that  is  upward,  downward,  or  bi-direc- 
tional. Figure  5 illustrates  three  phases  of 
storm  dynamics  contained  in  cells  S,,  S.,  Sg. 

A moving  aircraft  can  experience  any  of  these 
profiles  while  in  the  storm  area.  As  the 
aircraft  enters  another  cell,  a typical  pro- 
file change  is  illustrated. 

LIGHTNING 

Thunderclouds  contain  a heterogenous 
array  of  charge  formations.  Many  theories 
suggest  that  charge  buildup  is  due  to  the 
driving  energy,  or  a wind  intensity  within 
the  thundercloud.  The  net  effect  is  that  a 
charge  differential  builds  up  both  within 
the  storm  cell  and  also  between  the  cell  and 
earth's  surface.  Our  interpretation  of  this 
phenomena  is  that  lightning  occurs  when  wind 
intensity  levels  or  successive  intensity 
level  changes  exceed  a predetermined  level. 

Reference  is  made  to  Figure  6 illustrat- 
ing the  logic  that  provides  a visual  light- 
ning cue  as  well  as  a radio  aural  cue.  The 
present  wind  intensity  (at^)  *s  compared  to 
the  previous  intensity  If  the 

intensity  level  difference  exceeds  a prede- 
termined level  reference,  then  lightning, 
thunder,  and  radio  noise  cues  are  provided. 

ICING 

In  areas  where  individual  thunderstorms 
are  isolated,  or  scattered,  icing  is  not  con- 
sidered to  be  a serious  problem.  However  in 
areas  of  numerous  thunderstorms,  the  icing 
problem  can  be  significant.  Therefore, 
thunderstorm  icing  discretes  for  icing  simu- 
lation should  be  provided  within  severe  storm 
line  modules  for  altitudes  above  the  order  of 
20,000  feet. 

CLOUD  SIMULATION 

When  the  trainer  aircraft  enters  a storm 
cell,  or  storm  line,  a cloud  cover  cue  is  sent 
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to  the  Visual  System.  Conversely,  when  the 
trainer  aircraft  leaves  the  storm  cell,  or 
storm  line,  a clear  view  cue  is  sent  to  the 
Visual  System.  The  cloud  cover  cue  is 
associated  with  the  storm  intensity. 

SIMULATION  BLOCK  DIAGRAM 

Reference  is  made  to  Figure  7 which 
shows  a typical  Thunderstorm  Simulation  Block 
Diagram. 

Storm  cell  and  storm  line  data  are 
instructor  furnished.  These  are  generally 
selected  apriori. 

Aircraft  equations  of  motion  are 
supplied  from  another  source.  The  items  of 
interest  involve  own  ship  location  and  alti- 
tude. The  location  defines  whether  the  storm 
volume  has  been  intercepted.  The  ship's 
altitude  is  required  to  define  wind  compo- 
nents. 

Two  random  noise  generators  are 
required  to  provide  the  storm  amplitude  and 
time  periods.  All  cells  within  the  storm 
front  can  be  activated  by  the  same  two  random 
generators.  The  individual  cell  characteris- 


tics will  differ,  however,  in  intensity  and  1 

polarity. 

Data  base  provides  the  time  references 
needed  for  various  phases  of  the  program. 

The  computed  functions  of  the  program 
interface  with  own  ships  equations  of  motion 
in  the  case  of  wind  intensity  or  icing. 

Visual  cues  are  provided  by  lightning  dis- 
cretes. Scatter  and  storm  distance  informa- 
tion is  also  provided.  Aural  cues  are  pro- 
vided by  thunder  and  radio  noise  discretes. 


References:  "The  Flight  of  the  Thunderbolts", 
Sir  Basil  Schonland,  Second  Edi- 
tion, Clarendon  Press,  Oxford, 
1964. 

"The  Lightning  Book",  P.  E. 
Viemeister,  Doubleday  & Co., 

1961. 

"Aviation  Weather  For  Pilots  and 
Flight  Operation  Personnel", 

FAA,  Dept,  of  Commerce,  1965. 
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FI i ght  Path  Vector 
in  Storm  Call  Coordinates 


Figure  2 Planar  View  of  Typical  Storm  Cell 
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INTENSITY 


Figure  5 Three  Simultaneous  Representations  of  Storm  Cell  Intensities 
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STIMULATION.  NOT  SIMULATION: 

AN  UPDATE  OF  IN-THE-FLEET  TRAINING 


J.  WHITNEY  and  T.  ATHNOS 
General  Physics  Corporation 


SUMMARY 

Training  in  the  art  of  Anti-Submarine  Warfare 
(ASW)  in  the  U.S.  Navy  has  not  been  totally 
neglected.  Yet,  neither  has  it  been  conducted  with 
sufficient  frequency  to  produce  highly  skilled  ASW 
attack  teams  able  to  cope  with  today's  major 
threat:  the  Soviet  submarine  force.  ASW  is  mainly 
a time-consuming  matter  of  attrition,  in  which 
numbers  count  more  as  friendly  losses  mount. 
Some  authorities  conclude  that  the  U.S.,  at  most, 
might  sink  10  percent  of  all  Soviet  submarines 
before  they  took  a serious  toll  among 
merchantmen. 

The  U.S.  Navy's  active  submarine  force,  as 
recently  as  February,  1977,  was  out-numbered  by 
the  Soviet  active  submarine  force  322  to  114.  This 
quantitative  shortage  could  prove  crucial  in  a 
showdown  with  the  Soviet  Union,  since 
"many-on-onef  seems  to  be  Moscow's  rule.  Although 
ASW  missions  are  shared  between  surface, 
subsurface  ^nd  air  platforms,  and  the  U.S.  holds 
qualitative  superiority,  U.S.  attack  submarines  face 
distinct  disadvantages  trying  to  check  three  times 
their  number. 


The  skill  level  of  ASW  attack  teams  needs  to  be 
improved.  Accomplishment  of  this  goal  can  be  met 
by  training  and  subsequently  verified  by  assessment. 
These  methods  have  been  used  in  the  past  but  not 
to  a level  sufficient  to  produce  experts.  The  Navy 
has  provided  simulators  and  stimulators  both  in  the 
Fleet  (incorporated  into  or  adjunct  to  the 
operational  equipment)  and  in  shore-based  training 
facilities.  Realism  has  been  lacking. 

Realism  in  ASW  training  does  exist  albeit 
infrequently.  Submarine  services  to  the  surface  force 
are  scheduled  on  a not-to-interfere  basis  with 
operational  necessities.  A further  problem  lies  in 
the  fact  that  scheduling  must  be  accomplished 
across  lines  of  operational  control;  eg., 
COMNAVSURFLANT  and  COMSUBLANT. 
Submarine  services  for  another  submarine  are  much 
easier  to  schedule  since  a Submarine  Squadron 
Commander  may  order  two  or  more  of  his  boats 
to  conduct  Inter-Ship  Exercises  (ISE).  This 
scheduling  can  be  accomplished  with  little  or  no 
need  for  Type  Commander  interaction. 

Other  means  of  realistic  training  occur  when 
encountering  targets  of  opportunity.  Granted, 


TABLE  I.  US/USSR  SUBMARINE  FORCES  (FEBRUARY  1977) 


U.S. 

USSR 

TYPE 

A 

B 

C 

TYPE 

A 

B 

C 

; SSBN 

J3 

6 

SSBN 

62 

6 

SSN 

61  i 

27 

2 

SSB 

22 

ss 

10 

SSGN 

41 

2 

SG 

1 

SSG 

28 

SSN 

39 

2 

SS 

1J0 

2 

100+ 

Ilk 

33 

3 

322 

121 

100+ 

150 

434+ 

A - ACTIVE 

B - UNDER  CONSTRUCTION 
C - RESERVE 
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exercise  weapons  cannot  be  launched  but  detection, 
tracking,  and  attack  solution  procedures  can  be 
exercised.  Oftentimes  a ship's  mission  precludes  the 
opportunity  to  conduct  meaningful  training  when 
chancing  on  these  targets. 

Operational  Readiness  Inspections  (ORI)  are 
conducted  on  all  U.S.  Navy  combatants  and 
auxiliaries.  Among  other  facets,  these  ORI's  are 
designed  to  assess  the  readiness  of  the  ship  for 
ASW.  The  surface  Fleet's  ORI's  are  conducted  at 
such  places  as  Guantanamo  Bay,  Cuba  (GITMO), 
San  Diego,  and  even  overseas.  Again,  the  availability 
of  submarine  services  is  a limiting  factor  in  the 
assessment  of  ASW  readiness.  It  is  not  uncommon 
for  a ship  to  satisfactorily  complete  the  ASW 
section  of  the  ORI  having  had  no  sonar  contact  on 
a real  submarine.  The  author  is  aware  of  at  least 
one  Chief  Sonar  Technician  with  12  years  of  Fleet 
experience  who  has  yet  to  observe  a submarine 
contact  on  his  sonar  equipment. 

Another  requirement  mandated  by 
COMNAVSURFLANT  is  that  all  AN/SQR-17  Passive 
Sonar  Operators  complete  20  hours  of  training  per 
month.  Presently,  due  to  the  lack  of  submarine 
services  and,  as  necessitated  by  the  equipment,  a 
lack  of  air  services  to  provide  sonobuoys  and  data 
links,  the  training  consists  of  "publication-reviewing" 

II 

and  use  of  the  Roof-Top  Trainer  # The  latter 
device  lacks  real-time,  quantitative  assessment 
methods,  must  be  used  while  in-port,  and  provides 
unrealistic  submarine  data. 

This  paper  will  present  a training  and  assessment 
program  designed  and  developed  by  General  Physics 
Corporation  to  overcome  deficiencies  in  ASW 
training. 

BACKGROUND 

ASW  training  using  simulators  and  stimulators 
was  recognized  as  necessary  only  20  years  ago. 
Then,  as  now,  submarine  services  to  provide 
realistic  ASW  training  to  surface  ships  was 
conducted  infrequently.  Tracking  targets  of 
opportunity  and  practicing  attack  solution 
procedures  on  them  were  the  only  means  by  which 


a surface  ship  could  maintain  peak  efficiency.  Most 
sonar  equipments  prior  to  1960  had  no  means  by 
which  to  be  stimulated  with  simulated  target 
signals. 

Classification  of  sonar  contacts  can  be 
accomplished  accurately  only  by  experienced 
personnel.  Training  in  this  subject  is  essentially  a 
matter  of  rote  learning.  Audio  tapes  furnish  aural 
data  on  the  various  types  of  underwater  targets. 
Examples  are  kelp,  marine  life,  pinnacles,  torpedoes, 
and  various  types  of  diesel  and  nuclear  submarines. 
Visual  classification  training  incorporates 
photographs  of  targets  at  various  aspects  and  speeds 
as  presented  on  the  sonar  display.  Again, 
memorization  is  required.  The  Performance 
Monitoring  Equipment  (RME),  used  in  conjunction 
vwth  later  models  of  the  AN/SQS-23  Sonar,  is  the 
only  stimulator  which  provides  realistic  active  sonar 
training.  Using  a 96  track  tape  recorder,  raw  target 
data  can  be  inserted  into  the  system  to  provide 
classification  and  tracking  training.  One  deficiency 
of  this  system  is  that  most  equipment  controls 
must  be  preset  negating  training  of  operators  on 
equipment  control  manipulation. 

Successive  active  sonars  of  the  AN/SQS-26  series 
were  designed  to  incorporate  Sonar  Monitors.  These 
requirements,  in  addition  to  monitoring  equipment 
transmit  and  receive  levels,  introduce  target  signals 
enabling  operators  to  receive  limited  training  in  the 
art  of  target  detection  and  tracking,  and  equipment 
operation.  Due  to  the  relatively  low-frequency 
spectrum  of  these  sonars  and  the  methods  used  to 
visually  display  targets,  classification  is  often 
determined  by  interpretation  of  target  motion 
rather  than  by  target  shape  or  target  echo  quality. 

One  of  the  newest  of  the  active  sonars  is  the 
Independent  Variable  Depth  Sonar  (IVDS).  This 
system  incorporates  a stimulator;  however,  it  is 
designed  to  provide  stationary  target  signals  for  test 
purposes. 

General  Physics  Corporation  has  devised  a 
methodology  whereby  these  stimulators  can  be 
programmed  to  provide  both  target  detection  and 
tracking  and  equipment  control  manipulation 
training  on  a maneuvering  target. 
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APPROACH 


Realizing  the  difficulties  encountered  by  the 
Fleet  with  regard  to  ASW  training  and  assessment. 
General  Physics  Corporation  considered  various 
methods  and  approaches  to  overcome  these 
problems.  The  final  decision  was  to  approach  the 
problem  in  the  following  manner: 

Use  existing  on-board  equipment  (no  new 
hardware). 

Use  "f  i x ed- 1 r a ck"  exercises  to  provide 
standardized,  quantitative  problems  for  assessment 
purposes. 

Use  realistic  scenarios  and,  in  the  case  of  passive 
sonar,  real  "at-sea"  ASW  encounter  data. 

Provide  exercises  categorized  into  various  levels 
of  complexity  for  both  individual  and  team 
training  and  assessment. 

Provide  a software-only  package  which  will 
provide  immediate  reinforcement. 

Develop  exercises  to  incorporate  all  phases  of 
ASW  operation  from  search  and  detection  to 
tracking  and  weapon  launch. 


Figure  I.  Surface  ASW  Operational 
Readiness  Assessment  and 
Training  (ORATS)  System 


From  these  general  guidelines,  the  Surface  ASW 
Operational  Readiness  Assessment  and  Training 
System  (ORATS)  was  developed.  ORATS,  then,  is  a 
program  for  measuring  individual  and  team 
performance  on  realistic  operational  problems  using 
installed  equipment. 

Each  of  the  exercises  in  ORATS  can  be  used  by 
a ship's  ASW  team  for  either  training  or  assessment 
purposes  based  on  the  capabilities  of  ASW 
personnel. 

Positive  Guidance  — Instructional  assistance  is 
provided  to  the  operator(s)  as  required  during 
the  exercise.  The  exercise  can  be  stopped  as 
necessary  to  explain  concepts  and  techniques. 
The  exercis-n  is  not  scored  since  assistance 
provided  by  the  Test  Director  invalidates  the 
results. 

Direct  Exposure  — The  exercise  is  conducted 
with  no  assistance  from  the  Test  Director.  The 
exercise  is  usually  scored  and,  using  the 
comments  and  values  recorded  during  the 
exercise,  thoroughly  critiqued  indicating  to  the 
operator(s)  areas  of  weakness  where  additional 
study  and  practice  is  required. 

Assessment  — The  exercise  is  conducted  in  the 
same  manner  as  during  Direct  Exposure.  Scoring 
is  mandatory.  The  assessment  method  provides 
commanding  officers  and  operational  unit 
commanders  a standardized  and  quantitative 
determination  of  the  operational  readiness  of 
their  crew(s). 

Exercise  scores  have  not  been  normalized,  nor  are 
average  scores  presently  available.  Scores  are 
meaningful  when  compared  to  previous  scores  of 
the  same  exercise  or  when  identifying  strong  and 
v\eak  areas  of  concern. 
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METHODOLOGY 

Surface  ASW  ORATS  is  being  developed  to 
provide  training  and  assessment  for  ASW  personnel 
in  the  Combat  Information  Center  (CIC),  Sonar 
Control,  and  Underwater  Battery  (U.B.)  Plot. 
Initially,  this  system  is  designed  only  for  the 
FF-1052  Class  Frigate  which  typically  has  2 active 
sonars,  1 passive  sonar,  torpedo  tubes,  and  an 
Anti-Submarine  Rocket  (ASROC)  launcher.  Due  to 
weapon  and  launcher  security  considerations,  all 
weapon  launcher  functions  are  either  simulated  or  a 
test  set  is  used  to  stimulate  the  UB  Plot  Attack 
Console. 

CIC  Exercises  (Figure  2.) 

Plotting  and  tracking  targets  in  CIC  requires  the 
use  of  either  the  NC-2  Plotter  or  the  Dead 
Reckoning  Tracer  (DRT).  The  NC-2  Plotter  can  be 
stimulated  with  own  ship  and  various  air,  surface 
and  subsurface  target  data  whereas  the  DRT  is 
stimulated  only  by  own  ship  motion  data.  As  an 
option,  target  data  stimulation  can  be  provided 
aurally  by  a radar/sonar  operator  via  sound-powered 
telephones  and/or  an  audio  tape  cassette. 

In  a typical  CIC  Functional  Team  Exercise  (CIC 
personnel  only,  with  external  inputs  simulated),  an 
audio  tape  will  provide  pre-exercise  instructions 
such  as  sonobuoy  field  position,  status  of  LAMPS 


ASW  helicopter  and  position  of  assist  ship.  During 
the  fixed  track  exercise  predetermined  own  ship 
and  target  maneuvers  are  provided  via  the  audio 
tape,  and  LAMPS  and  assist  ship  positions  on  a 
minute-by-minute  basis  stimulate  the  NC-2  Plotter. 
Assessment  (or  training!  of  the.  Plotter  Operator 
and  CIC  Evaluator  is  accomplished  by  determining 
the  very  detailed  completeness  and  accuracy  of 
various  plotting,  tracking,  and  target  data 
interpretation  functions. 

Passive  Sonar 

The  FF-1052  Class  Frigate's  passive  sonar 
capability  accepts  actual  target  data  only  from 
sonobuoys  via  support  aircraft  radio  link. 
Sonobuoys,  each  at  a different  frequency,  are 
launched  from  an  airborne  platform,  then 
monitored  by  that  platform.  The  aircraft  selects 
one  or  more  sonobuoy  signals  for  transmission  to 
the  Frigate.  Each  Frigate  is  equipped  with  a 
1-inch  tape  recorder  to  oreserve  the  data  for 
further  analysis. 

The  General  Physics  analysts  have  collected  and 
thoroughly  analyzed  a series  of  actual  Soviet-U.S. 
at-sea  encounters.  These  recordings,  used  to 
stimulate  the  Frigate's  passive  sonar  during  ORATS 
exercises,  provide  one  of  the  few  realistic, 
nonsimulated  training  and  assessment  programs 
available  to  the  Fleet. 


Figure  2.  CIC  Exercise  Equipment  Lineup 
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Passive  sonar  used  onboard  surface  ships  is 
relatively  new.  Therefore,  some  operational  doctrine 
is  non-existent.  Equipment  setup  and 
communication  procedures  vary  from  ship  to  ship. 
The  purpose  of  ORATS  as  it  relates  to  passive 
sonar  training  is  not  to  set  doctrine,  but  to  provide 
procedures  the  Passive  Sonar  Operator  can  use  to 
enable  him  to  detect  more  targets,  classify  targets 
more  accurately,  and  provide  more  relevant 
information  such  as  tactical  clues  to  the  CIC 
Evaluator. 

A typical  passive  sonar  exercise  begins  by 
informing  the  operator  of  the  threat  type  (e.g., 
possible  Soviet  Type  II  nuclear  submarine)  and 
threat  level.  The  1-inch  tape  stimulates  the 
equipment,  providing  realistic  target  data.  The 
operator  is  required  to  identity  the  frequencies 
emanating  from  the  threat,  closest  points  of 


approach  (CPA's)  to  various  sonobuoys,  threat 
course  and  speed,  and  threat  classification.  He  is 
also  required  to  provide  useable  and  relevant 
information  to  the  CIC  Evaluator. 

Active  Sonar  (Figure  3.) 

The  AN/SQS-26CX  Sonar  System  is  installed  on 
all  FF-1052  Class  Frigates.  The  AN/SQS-35(V) 
system,  an  Independent  Variable  Depth  Sonar 
(IVDS),  is  installed  on  most  of  this  class  Frigate. 

The  content  of  the  ORATS  System  is  determined 
by  ship  class;  however,  those  ships  not  having  the 
IVDS  System  will  still  be  able  to  use  all  but  two 
exercises. 

Each  of  the  active  sonars  incorporates  a target 

simulator.  These-  units  can  simulate  own  ship's 

movement  in  addition  to  a single  target.  Again, 


Figure  3-  Active  Sonar  Exercise  Equipment  Lineup 
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fixed  track  exercises  are  used  to  cause  stimulation 
of  sonar  search  and  track  consoles.  Targets  can  be 
maneuvered  to  simulate  movement  of  actual  targets 
and  can  be  positioned  to  require  the  operators  to 
use  the  various  available  equipment  operating 
modes. 

A typical  ORATS  exercise  to  train  or  assess  a 
functional  team  (one  or  both  sonars)  provides  a 
target  at  long  range.  The  Search  Console  Operator 
is  required  to  detect  the  target  and  manipulate 
equipment  controls  to  enable  the  Target  Tracking 
Console  Operator  to  assume  tracking  responsibility. 
The  Sonar  Supervisor  oversees  these  actions  and,  in 
addition,  is  required  to  interpret  and  report  target 
data  in  accordance  with  standard  doctrine.  The  Test 
Director  simulates  responses  from  stations  external 
to  Sonar  Control.  The  Search  Console  Operator  is 
not  required  to  determine  target  classification  since 
the  signal  which  stimulates  his  equipment  is 
qualitatively  unrealistic. 


AN/SQS-26CX 


-«J 


Underwater  Battery  (U.B.)  Plot  (Figure  4.) 

Training  and  assessment  of  UB  Plot  personnel 
must  be  conducted  in  concert  with  training  and 
assessment  of  Active  Sonar  personnel.  With  regard 
to  target  data,  the  Underwater  Battery  Fire  Control 
System  (UBFCS)  can  be  stimulated  only  by  sonar 
or  radar  inputs.  Weapon  and  launcher  test  sets  will 
provide  UBFCS  stimuli  which  provide  weapon  and 
launcher  status  indications. 

The  function  of  the  UBFCS  is  to  receive  raw 
target  data  from  either  of  the  active  sonars  or  a 
fire  control  radar.  These  data  are  presented  such 
that  an  operator  can  determine  target's  course  and 
speed.  When  a weapon  and  launcher  are  selected, 
the  UBFCS  solves  the  ballistic  computations  and 
remotely  controls  the  weapon  and  launcher  for 
optimum  firing  time  and  aim  point. 


AN/SQS-35 (V) 
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Figure  4.  Active  Sonar/UB  Plot  Exercise  Equipment  Lineup 
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As  previously  discussed,  weapon  security 
considerations  preclude  use  of  the  ASROC  launcher 
and  even  the  associated  test  set  which  can 
stimulate  the  UBFCS.  Exercises  calling  for  ASROC 
launches  are  conducted  by  requiring  the  Test 
Director  to  make  announcements  simulating  ASROC 
launcher  and  weapon  indicator  responses.  Simulated 
torpedo  tube  attacks  can  be  conducted  as  above  or 
by  using  a test  set  to  stimulate  the  UBFCS  weapon 
and  launcher  indicators.  The  latter  method  is 
preferred  but  often  requires  excessive  labor  to 
unload  the  torpedo  tubes  in  addition  to  requiring 
the  presence  of  three  additional  personnel  to  run 
the  exercise. 


The  Surface  ASW  ORATS  Program  for  the 
FF-1052  Class  Frigate  is  comprised  of  three  sets, 
each  consisting  of  25  exercises.  Four  levels  of 
complexity  are  designed  into  the  Program. 

Level  I - Combined  ASW  Team 
(1  exercise  per  set) 

The  combined  Sonar/UB  Plot  and  CIC  Teams 
interact  to  prosecute  simulated  submarine  targets 
in  a realistic,  operational  scenario.  Level  I 
exercises  involve  all  functional  skills  using  all 
equipment.  It  takes  about  2 hours  to  run  this 
exercise. 


A typical  Combined  Skill  Team  exercise  trains  or 
assesses  both  the  active  sonar,  as  described 
previously,  and  the  UB  Plot  personnel.  UB  Plot 
personnel  training  and  assessment  is  based  on  their 
ability  to  properly  operate  the  UBFCS  equipment, 
accurately  interpret  target  data,  select  proper 
weapons  and  launchers,  and  conduct  an  accurate 
and  timely  attack.  Communications  between 
stations,  some  of  which  are  simulated,  is  also  an 
important  factor  which  is  considered  in  scoring  of 
the  exercise. 

The  four  ASW  stations  described  above  always 
work  together  when  prosecuting  a real  target. 
However,  seldom  do  all  stations  have  the  same 
degree  of  expertise.  To  counter  this  disparity, 
ORATS  has  been  designed  to  provide  training  or 
assessment  of  individual  operators,  separate 
functional  teams,  various  combinations  of  functional 
teams,  or  the  entire  ASW  attack  team. 


Figure  5-  ORATS  Program  Structure 


Level  II  — Combined  Skill  Team 
(4  CIC  exercises  per  set) 

(2  Active  Sonar/UB  Plot  exercises  per  set) 

The  Sonar/UB  Plot  Team  or  the  ClC  Team 
individually  prosecutes  simulated  submarine 
targets  in  a realistic,  operational  scenario. 

Stations  not  manned  are  simulated  by  the  Test 
Director.  These  typically  take  1 to  1 VS  hours, 
including  scoring  and  critique. 

Level  III  — Functional  Team 
(2  CIC  exercises  per  set) 

(2  Active  Sonar  exercises  per  set) 

(1  Passive  Sonar  exercise  per  set) 

A specific  Sonar  Team  or  the  CIC  Team  is 
exercised  during  prosecution  of  a simulated 
submarine  target.  Only  the  equipments  required 
for  the  skill  being  evaluated  are  used.  External 
stations  not  manned  are  simulated  by  the  Test 
Director.  These  take  about  1 hour. 

Level  IV  — Individual  Skill 
(7  CIC  exercises  per  set) 

(4  Active  Sonar  exercises  per  set) 

12  Passive  Sonar  exercises  per  set) 

Individual  operators  are  evaluated  with  regard  to 
specific  skills  on  a particular  equipment  or  unit 
of  a system.  These  take  about  45  minutes  to  an 
hour. 

1 


173 


TAPES 

• AUDIO  CASSETTE  EXERCISE 
INSTRUCTIONS  AND  NARRATIVES 

• 1- INCH  PASSIVE  SONAR 
DATA  TAPE 


TEXTS 

VOLUME 

1 

PROGRAM  MANUAL 

VOLUME 

1 1 

CIC  EXERCISES 

VOLUME 

1 1 1 

PASSIVE  SONAR  EXERCISES 

VOLUME 

IV 

ACTIVE  SONAR  EXERCISES 

VOLUME 

V 

ACTIVE  SONAR  EXERCISES 

Figure  6. 

SOFTWARE  FORMAT 

General 

The  ORATS  Program  delivered  to  the  ship 
consists  of  five  looseleaf  binders,  one  1-inch  tape 
for  the  Passive  Sonar  recorder-playback  device,  and 
17  audio  cassette  tapes.  Stopwatches  and  a cassette 
tape  playback  device  must  be  furnished  by  each 
ship  to  enable  them  to  run  any  ORATS  exercise. 

Volume  I,  Program  Manual,  designed  for  the 
ship's  Training  Officer,  contains  introductory 
material  and  a Selection  Index  summarizing  all 
exercises  in  the  set.  This  volume  is  unclassified. 

Volume  II,  CIC  Exercises,  designed  for  the 
Operations  Department,  contains  all  Level  II 
through  Level  IV  CIC  exercises  and  the  Level  I 
Combined  ASW  Team  exercise.  This  volume  is 
classified  Secret. 

Volume  III,  Passive  Sonar  Exercises,  designed  for 
the  Weapons  Department,  contains  all  Level  III  and 
IV  passive  sonar  exercises.  This  volume  is  also 
classified  Secret. 

Volumes  IV  and  V,  Active  Sonar  and  Combined 
Active  Sonar/UB  Plot  Exercises,  also  designed  for 
the  Weapons  Department,  contain  all  Level  II 
through  IV  Active  Sonar  and  Combined  Active 
Sonar/UB  Plot  exercises  These  volumes  are 
classified  Confidential. 


ORATS  Program  Package 

The  1 -inch  Passive  Sonar  tape  is  classified  Secret, 
as  are  about  half  of  the  audio  cassette  tapes.  The 
remainder  of  the  cassette  tapes  are  classified 
Confidential. 

Exercise  Format 

Each  ORATS  exercise  is  divided  into  five  basic 
sections. 

Content  Sheet  — The  first  page  of  each  exercise 
provides  a combination  index/list  of  effective 
pages  which  is  required  for  classified  material.  A 
very  brief  summary  of  pre-exercise  requirements 
is  also  presented  which  lists  essential  equipment. 

Exercise  Summary  — This  section  provides  an 
abstract,  a brief  scenario,  a list  of  equipment 
and  personnel  required,  exercise  time,  the  basis 
of  evaluation,  and  a chronological  summary  of 
events  including  a track  chart  plot.  (Figure  7.) 

Procedure  — The  Test  Director  uses  this  section 
to  conduct  and  oversee  the  progress  of  the 
exercise.  The  Procedures  describe  switch  line-ups, 
equipment  connections,  and  specific  actions  to 
be  accomplished  by  the  Test  Director,  the 
Observers  and  the  personnel  being  evaluated  on  a 
minute-  by-  minute  basis.  It  provides  specific 
evaluation  instructions  including  quantitative 
performance  standards,  standardized  checkpoints 
and  space  to  record  data  and  score  the  results. 
(Figure  8.) 
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TEST  PROCEDURE 


TEST  NUMBER 


TITLE 

FUNCTIONAL  TEAM  TEST 
SONAR-ACT IVE-AN/SQS- 35  (V) 

Time 

hrmin:sec 

Test  Instructions 

Data 

Evaluation  Guide 

Ref 

0:32:00  TEST  DIRECTOR: 


Announce,  "All  stations,  Bridge.  We  are  coming  left  with  standard 
rudder  to  course  335  and  increasing  speed  to  15  knots." 


0:32:00  TEST  TARGET  SIMULATOR  OPERATOR: 

Make  the  following  settings: 

Own  Ship's  course  left  to  335  at  a rate  of  2 degrees  per  second. 
Own  Ship's  speed  to  15  knots. 


0:33:00  TEST  DIRECTOR: 

Does  the  Sonar  Supervisor  direct 
the  Operators  to  search  sector 
1 80  to  270? 

Does  the  Unit  I Operator  select 
SCAN  RECEIVER? 

Does  the  Unit  2 Operator  select 
maximum  range  scale? 


0:3*1:00  TEST  DIRECTOR: 

Does  the  Unit  1 Operator  detect 
the  target? 

By  time  0:3*t:00? 

Does  the  Unit  I Operator  report: 

Target  detection  to  the  Sonar 
Supervi sor? 

Bearing  drift? 

Range  drift? 

Target  Doppler? 


Yes 

Yes 

ves 

2 A 

2 B 

2 C 

Yes 

1 B 

Yes 

1 B 

Yes 

1 D 

Left 

1 D 

Closi ng 

1 D 

S 1 i ght  up 

1 D 

PAGE  NUMBER 


Figure  8.  Procedure  (example) 
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Test  Result  Summary  — A categorized  summary 
by  function  of  points  scored  during  the  exercise 
is  recorded  in  this  section  then  compared  to  the 
maximum  score  possible.  The  resultant  percentage 
of  each  category  can  be  used  as  a means  to 
determine  areas  of  weakness  in  which  further 
training  is  required.  Scores  have  not  bepn 
normalized  nor  is  there  any  basis  from  which  a 
pass/fail  grade  can  be  determined.  As  these 
exercises  are  used,  the  Fleet  percentile  will  be 
obtained,  thereby  enabling  evaluators  to  better 
measure  team  and  individual  ability.  This 
unclassified  section  can  be  used  for  inclusion  in 
department  training  records.  (Figure  9.) 

Appendices  — Each  Observer  is  furnished  an 
exercise  appendix  which  provides  data  extracted 
from  the  Procedure  section.  Format  and  use  is 
identical  to  the  Procedure;  however,  only  those 
sections  of  the  Procedure  pertaining  to  that 
particular  Observer  is  included.  Other  appendices 
provide  such  information  as  tactical  and 
environmental  data,  equipment  transmission 
checks  (which  ensure  proper  alignment),  reference 
notes,  and  a listing  of  the  Personnel 
Qualification  Standards  (PQS)  satisfied. 

CONCLUSIONS 

Surface  ASW  OR  ATS  evolved  from  a program 
developed  by  General  Physics  Corporation  for  the 
U.S.  Navy  submarine  force:  Submarine  Operational 
Readiness  Assessment  and  Training  (SORAT) 
Program.  SORAT  has  proven  practical  and  effective 
in  the  training  and  reinforcement  of  submariners' 
ASW  skills.  Submarine  crews  are  using  SORAT  for 
training  to  prepare  for  predeployment  inspections, 
tactical  weapon  certification,  and  other  readiness 
assessments. 

ORATS  is  presently  being  implemented  on  all 
FF-1052  Class  Frigates.  On  completion  of  the 
16-hour  demonstration,  ship  crews  most  always 
respond  with  plaudits  for  the  Program.  Typical 
responses  refer  to  the  need  for  such  a program,  its 
administrative  simplicity  and  all-encompassing 
assessment  of  operators  and  teams,  and  finally  its 
maintenance-free  aspect. 

ORATS  will,  in  all  probability,  be  developed  for 


other  ship  classes,  such  as  the  DD-963  Class 
Destroyer.  This  class  vessel  incorporates  the  Navy 
Tactical  Data  System  (NTDS)  with  an  ASW 
program  requiring  a more  complex  form  of  the 
ORATS  Program.  Similar  work  is  already  under 
contract  for  the  SSN-688  Class  computerized 
AN/BQQ-5  System.  The  *ormat  will  be  similar  to 
the  FF-1052  Progrrm,  yet  will  include  more 
tactically  significant  features  as  mandated  by  the 
additional  equipments  installed. 

Investigations  are  being  conducted  to  determine 
the  feasibility  of  development  of  an  ORATS-type 
training  program  for  use  by  shore-based  ASW 
trainers  and  an  assessment  program  for  use  by 
Refresher  Training  (ORI)  Inspectors.  This  type 
program  can  be  developed  for  most  any  equipment 
which  has  the  capability  of  being  stimulated.  A 
similar  program  has  been  demonstrated  to  the  Naval 
Aviation  ASW  community.  Anti-Air  Warfare  (AAW) 
Teams  are  likely  candidates  for  an  ORATS  training 
and  assessment  program.  A similar  program  has 
been  incorporated  into  computerized  simulators.  The 
computer  functions  as  Test  Director  and  Observer, 
recording  and  scoring  all  control  manipulations.  In 
this  type  format,  a human  Observer  would  be 
required  only  to  record  nonmechanical  operator 
functions. 

Training  and  assessment  practices  must  be  based 
on  objective  and  quantitative,  not  subjective  and 
qualitative  judgments.  Programs  must  be  developed 
in  a cost-effective  manner.  Stimulating  existing 
equipments  when  possible  is  certainly  less  costly 
than  producing  new  and  more  sophisticated 
simulators.  Maintenance  and  manpower  costs  must 
be  considered  in  addition  to  initial  procurement 
costs.  The  ORATS  Program  and  its  relatives  are 
cost-effective,  objective,  quantitative,  and,  above  all, 
effective  in  the  improvement  of  ASW  skills. 
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A NEW  APPROACH  FROM  IMPROVING  TACTICAL  ACTION/REACTION  TRAINING  AND  TACTICAL  PROCEDURES 
DEVELOPMENT  USING  THE  ACTION  SPEED  TACTICAL  TRAINER  IN  THE  GERMAN  NAVY 


CDR.  JURGEN  PROCHNOW  AND  LCDR.  BERND  LEHMANN 
Naval  Tactical  Training  Group 
Wilhelmshaven  Ebkeriege 
Germany 


I.  INTRODUCTION 

In  the  late  sixties  the  development  of 
technology  for  naval  warfare  came  into  a 
dynamic  phase  by  new  innovations  in  the 
area  of  electronic  warfare  as  well  as  by  the 
introduction  of  surface-to-surface  missiles 
and  the  resulting  need  to  provide  defensive 
measures  against  them. 

The  well-known  interdependence  of  tech- 
nology and  tactics  required  the  adaption  of 
naval  warfare  to  these  new  technical  aspects. 
The  sinking  of  the  Israeli  destroyer 
"Eilath"  indicated  this  very  clearly. 

The  influence  on  tactics,  provoked 
by  progress  in  technology  of  sensors  and 
effectors,  forced  consideration  of  improved 
tactical  training  for  officers.  Up  to  that 
time  trainers  - operating  via  an  electro- 
mechanical system  - had  been  considered  to 
be  sufficient  for  this  purpose.  But  it 
became  evident,  that  the  training  devices 
in  their  operating  concept  had  to  allow  for 
this  interdependence. 

Within  the  German  Navy,  the  following 
tasks  are  assigned  to  the  Naval  Tactical 
Training  Group  (NAVTACTNGGP)  in  Wilhelmsha- 
ven: 

- Improvement  of  the  tactical  training 
of  naval  and  naval-air-arm  officers. 

- Establishment  of  the  fundamentals  of 
naval  tactics  and  their  further  development. 

- Research  of  tactical  and  operational 
problems  in  specific  areas  of  naval  warfare 
which  are  considered  most  important  for  the 
German  Navy. 

- Further  development  of  naval  tactics 
concerning  the  adjustment  to  the  technologi- 
cal progress  and  formulation  of  appropriate 
procedures. 

At  the  above  mentioned  time  period,  the 
NAVTACTNGGP  had  the  Redifon-ASTT  in  service. 
This  trainer,  however,  due  to  its  technology, 
was  no  longer  sufficient  to  simulate  naval 
warfare  actions  in  a modern,  realistic  envir- 
onment. This  also  meant,  that  the  mission 
to  train  officers  in  tactical  decision  making 
was  very  difficult  to  accomplish. 

The  commissioning  of  the  preponderant- 
ly software-oriented  Singer-ASTT  as  the  re- 
placement of  the  Redifon-ASTT  in  March  1976 
provided  the  potential  to  simulate  all  as- 
pects of  modern  naval  warfare,  to  adapt  to 


future  technological  and  tactical  develop- 
ments, and  to  offer  enough  flexibility  for 
solving  tactical  problems. 

II.  THE  STRUCTURE  OF  THE  ASTT 

The  ASTT  consists  of  commercial  equip- 
ment delivered  by  several  sub-contractors 
with  Singer-Link  as  the  prime  contractor. 

The  prime  contractor  was  held  responsible 
for  Interfacing  components  into  an  integrated , 
fully  operational  hardware-software  complex. 

With  respect  to  the  tasks  of  this  simu- 
lator it  did  not  appear  necessary  to  adapt 
the  equipment  to  the  combat  information  cen- 
ters of  the  different  naval  assets,  but  in- 
stead to  employ  commercial  hardware  with 
software  developed  to  simulate  each  training 
scenario. 

A.  EXTENT  OF  THE  HARDWARE 

The  hardware  consists  of  the  following 
components: 

* The  nucleus  of  the  simulator  are  4 
free-programmable  computers  PDP  11/45  of  Di- 
gital Equipment  Corporation  with  the  neces- 
sary peripherals.  A total  core  of  300  K is 
available;  the  word-length  is  16  bit. 

* Each  of  the  14  cubicles  may  serve  as 
a submarine,  surface  ship,  or  aircraft.  They 
all  are  alike  equipped  with  a commander's  dis- 
play unit,  a horizontal  display  unit,  and  a 
communications  console  providing  external  (HF/ 
UHF)  and  internal  communications.  An  alpha- 
numerical  display  console  supplemented  by  a 
keyboard  is  used  as  part  of  each  of  these 
units/console.  Each  cubicle  is  under  direct 
control  of  a commanding  officer/pilot;  two 
cubicles  have  an  additional  horizontal  dis- 
play unit,  and  can  serve  as  units  of  the 
Officer  in  Tactical  _Command  (OTC).  The  above 
mentioned  video  situation  displays  present 

a synthetic  air/surface-,  and,  if  appropri- 
ate, a subsurface-picture,  consequently,  there 
is  a combination  of  all  information  available 
by  the  existent  sensors  of  the  respective 
vehicle. 

The  display  of  information  - similar  to 
that  of  the  command  and  control  systems  of 
the  German  destroyers  and  missile-carrying, 
fast  patrol  boats  in  combination  with  the  pic- 
ture-compilation via  LINK  11  data-transfer  in 
a force  - is  automatically  provided.  Naviga- 
tional details  as  well  as  jamming  are  also  si- 
mulated on  the  display. 
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4 3 2 


1 Commander's  console  5 Communications 

2 Horizontal  displov  console  6 E SM  indication  panel 

3 Communication  console  ^ Cubicle  data 

4 Chart  table  ® State  boards 


Each  cubicle  has  a defined  track-capa- 
city as  a function  of  its  personnel  capabil- 
ities. Evidently,  the  minimum  capacity  is 
assigned  to  a fighter  aircraft,  whereas  the 
maximum  capacity  is  provided  for  a unit  with 
an  automatic  command  and  control  system.  Dif- 
ferent delay  times  are  implemented  to  simulate 
the  distinct  transmission  times  as  a function 
of  the  degree  of  automation. 

In  addition  to  the  tactical  display  unit, 
the  al phanumberical  display  terminal  provides 
much  of  the  essential  information  necessary  to 
assess  the  tactical  situation;  it  is  used  for 
selecting,  controlling,  and  monitoring  a large 
number  of  exercise  parameters.  The  informa- 
tion and  parameters  are  displayed  in  page 
formats,  referred  to  as  tableaus.  The  standard 
tableau  - see  Figure  2 - is  divided  into  five 
areas  as  shown  below: 

- The  top  three  lines  contain  the  cubi- 
cle's operating  parameters  (speed,  course,  al- 
titude/depth, and  position),  track  number,  the 
current  exercise  time,  and  the  name  and  mnemo- 
nics of  the  tableau. 

- The  left  center  portion  of  the  tableau 
is  provided  for  input  of  new  data. 

- The  center  portion  of  the  tab- 
leau is  provided  for  information  relevant  to 


Figure  1.  Cubicle  Layout 

ick-capa-  the  data  entry  area.  Readouts  of  current 


values  of  selected  data  are  also  provided  in 
this  area. 

- The  right  center  portion  of  the  tab- 
leau is  provided  for  alert  messages  like 
"ESM-Contact"  and  "Missile  Hit." 

- The  lower  portion  of  the  tableau  is 
provided  for  operator  data  entry. 


STANDARO 

tableau  heaoerarea 

DATA 

INFORMATION 

ALERT 

IN  PUT 

AREA 

AREA 

AREA 

DATA  INPUT  CHECK 

LINE 

Figure  2.  Standard  Tableau 

It  is  possible  to  modify  at  any  time  the 
operating  parameters  without  a change  of  a 
tableau. 

* A Maritime  headquarter  (MHQ)  permits 
the  involvement  of  the  operational  level  and 
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the  respective  decision-making  processes  in 
exercises.  The  MHQ  also  has  tactical  display 
units  with  al phanumberical  display  terminals. 

A Vertical  Display  Plot  (VDP)  is  included  in 
the  MHQ.  It  is  a monochrome  display  which  pre- 
sents an  overall  situation  display.  The  pre- 
sentation of  information  results  from  LINK  11 
or  conventional  data-transfer  by  the  cubicles. 

* An  auditorium  with  a seating  capacity 
for  60  persons  also  serves  for  exercise  control 
and  a subsequent  debriefing.  To  monitor  exer- 
cises, seven  control  consoles  are  available; 
each  is  fitted  with  a tactical  display  unit 
and  two  alphanumerical  display  units  supple- 
mented by  communications  equipment. 

* A Uirge  Screen  Display  (LSD)  is  in- 
cluded in  the  auditorium;  it  is  a color  system 
which  projects  an  overall  view  of  the  tactical 
situation  on  a screen,  with  matched  tape  record- 
ed exercises. 

B.  EXTENT  OF  THE  SOFTWARE 

As  already  pointed  out,  the  important 
difference  between  this  ASTT  and  the  Redifon 
ASTT  is  that  the  trainer's  performance  results 
from  the  inherent  capabilities  of  the  software 
implementation  which  consists  of  the; 


- operation/simulation  program,  and  the 

- training  assistance  program. 

1.  The  Operation/Simulation  Program 

This  program  consists  of-  several  modu- 
les which  control  the  entire  flow  of  exercise 
events  with  all  its  functions  in  real-time;  in 
particualar,  it  controls  the  vehicles  sensors, 
effectors,  communication-systems,  their  opera- 
ting parameters,  and  the  environmental 
conditions,  in  which  they  operate.  Via  the 
following  tableaus  the  vehicle's  capabilities 
and  information  necessary  for  its  employment 
are  controlled/directed: 


Damages  affected  by  weapon  engagements 
can  also  be  simulated.  The  program  offers  the 
possibility  to  display,  in  addition  to  the  14 
cubicles,  112  "synthetic"  targets.  The  pro- 
gram also  provides  the  access  to  the  extensive 
data  base  with  a collection  of  characteristics 
and  specifications  about  vehicles,  sensors  and 
effectors. 

a.  Simulation  of_Sensors 

Eacfj~cuETcTe~can~8TIpose  of  the  fol- 
lowing sensors  in  accord  with  the  particular 
type-/c lass-definition: 

- Radar  (up  to  4 sets), 

- Sonar  (up  to  3 sets), 

- Electronic  Support  Measures  - ESM  (up 
to  2 sets), 

- Magnetic  Anomality  Detection  (1  set), 

- Sonobuoys  (quantity  according  to  the 

type ) , 

- Infrared  Detection  (1  set),  and 

- Tactical  air  navigation  (1  set). 

The  functions  of  a particular  sensor  are 
simulated  only  as  necessary  for  tactical 
decision-finding.  For  instance,  the 
frequency,  the  transmission,  and  the 
scan-mode,  if  appropriate,  can  be  selected  for 
each  radar  set.  On  the  other  hand,  after  the 
detection  of  a hostile  radar,  its  band  or  even 
the  frequency,  the  bearing  accuracy,  and  the 
spot  number  will  be  displayed  on  a corres- 
ponding tableau  as  a function  of  the  quality 
of  the  utilized  ESM-set. 

b.  Simulation  of_Effectors 

Eacfi~cu5TcTe~can~3Tspose  of  the  follow- 
ing effectors  in  accord  with  the  particular 
type-/cl ass-definition: 

- Missiles  (up  to  3 systems), 

- Guns  (up  to  2 systems), 

- Torpedoes  (up  to  2 systems), 

- Rocket  torpedoes  (1  system), 

- Antisubmarine-Warfare  rockets  (up  to 
2 systems). 


IOX 

CUBICLE  INDEX 

VEH 

VEHICLE  CONTROL 

SON 

SONAR 

TGT 

TARGET  CONTROL 

SBV 

SONOBUOYS 

MTR 

MANUAL  TRACK 

AS  1 

ASW  INFORMATION 

1 OT 

IDENTIFICATION 

AAI 

ASW  AIR  INFORMATION 

TO  1-4 

TARGET  DATA 

SSN 

SPECIAL  SENSORS 

MNV 

MANEUVERS 

REF 

REFERENCE  POINTS 

WPS 

WEAPONS  SUMMARY 

MIS 

MISSILES 

RAD 

RADAR 

GUN 

GUNS 

COM 

CUBICLE  COMMUNICATIONS 

TOR 

TORPEDOES 

RAT 

RATT  COMMUNICATIONS 

AWP 

ASW  WEAPONS 

ESM 

ELECTRONIC  SUPPORT  MEASURES 

AIR 

AIR  WEAPONS 

EE  1 

EW/ESW  INFORMATION 

RKT 

RACKET  DATA 

DMG 

DAMAGE  ASSESSMENT 

ECM 

ELECTRONIC  COUN TERME ASURES 

Figure  3.  List  of  Tableaus 
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- Depth-charges  (quantity  according  to 
the  type), 

- Bombs  (quantity  according  to  the 

type), 

- Electronic  Countermeasures  - ECM  - 
(up  to  2 jammers  and  chaff-quantity  according 
to  the  type). 

For  instance,  the  model  for  the  missile 
employment  provides  the  following  features: 

- Selection  of  the  appropriate  system 
(SSM,  SAM), 

- Readiness  to  launch  the  missile(s) 
not  till  correct  insertion  of  relevant  data 
and  target  in  range,  and  open  arcs, 

- Selection  of  search-parameters, 

- Target  acquisition  if  a target  with- 
in search-area,  and 

- Reduction  of  hit-probability  if  mis- 
sile is  subject  to  adequate  ECM  by  the  target. 

This  simulation  model  comprises  all 
essential  criteria  which  are  substantial  for 
the  actual  missile  employment,  and  therefore 
also  significant  for  the  tactical  training  at 
the  ASTT.  Figure  4.  shows  the  relevant  table- 
au. Similar  features  are  provided  for  the 
other  effector  types,  if  appropriate.  The 
instructor  is  able  to  intervene  to  prevent 


a hit  or  to  enforce  one,  based  on  data  informa- 
tion only  available  to  him. 

c.  S i mul at ion_of_Communi cat ion_S^s terns 

TRe”ASTT*Ts  equTppe<TwTtF_3'3i T Ferent 

types  of  communication  systems,  namely,  the 
simulation  of 

- Automatic/Semiautomatic  data  trans- 
fer (LINK  11/14), 

- Radio  teletype  (RATT)  communications, 
and 

- Voice  communications. 

The  systems  are  also  activated  according 
to  the  particular  type  definition. 

d.  Simulation  of_Damages 

This  part  of  the  program  simulates  the 
effect  of  weapons  employed  to  carry  on  tacti- 
cal exercises  under  realistic  combat  situa- 
tions during  the  engagement  phase.  Inflicted 
damages  are  determined  by  a random  number  ge- 
nerator as  a function  of  the  employed  weapon 
and  attack  unit.  If  the  instructor  does  not 
override  such  a damage  message  by  an  appro- 
priate input,  the  indicated  damaged  systems  of 
the  affected  vehicle  are  automatically  turned 
off  or  modified:  even  a total  loss  is  possible. 


MIS  MISSILES  TN  K00 

2 COURSE  : 2SO  ( 280)  DEGR 

3 SPEED  0020  (0020)  KTS 


5 MISSILE  SYSTEM  : 3 ( 1-3  ) 
SYSTEM  1 
2 
3 

9 TARGET  TN:  0040  OR 

10  BEARING: 

11  DISTANCE:  YDS 


MISSILE  SYSTEM  SSM 
MISSILES  AVAILABLE  04 
MAX  RANGE  40  NM 

MIN  RANGE  5 NM 

GUIDANCE 

MIOCOURSE 

TERMINAL 


21  S3  20 
XPOS  5120 
YPOS  5120 


13  LAUNCHER  NO  : 1,2 

14  MISSILES  TO  FIRE  : 2 

15  FC  RADAR  : 1 

16  MIOCOURSE  HEIGHT  : 1 

17  TERMINAL  HEIGHT  : 1 
SETTING  TERMINAL  HOMING 

10  SEARCH  WINDOW  . 2 

20  A2IMUTH  : 2 

21  RANGE  GATE  ; 2 

2 2 ACTIVATION  RANGE:  2 


LAUNCHERS  4 
1 OR  2 

0 : OFF  ( 1 : ON 

1 : 0050  , 2 : 0050  , 3:0050  (YDS) 

1 : 0010,  2:0015,3:0025  (YDS) 

1 : WIDE  , 2 : NORMAL  , 3 : NARROW 
1 : 010,  2:  015,  3:025  ( DEG) 

1 : 02000  , 2 : 05000  , 3 : 09000  (YDS) 
1:  08000  , 2:  16000  , 3 : 21500  (YDS) 


READY 
25  FIRE  : 0 


1 : FIRE  , 2 : UNLOAD 


LINE  25  DATA  1 


Figure  4.  Missile  Tableau 
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The  effectiveness  of  the  employment  of 
sensors,  effectors,  and  communication  systems 
is  influenced  by  the  environmental  conditions 
in  the  operation  area.  This  consideration  can 
have  a cogent  impact  on  the  tactical  deci- 
sion-finding process;  therefore,  the  following 
conditions  are  taken  into  account  for  the  si- 
mulation: 

- Wind  (direction/speed), 

- Current  (di recti  on/ speed ) , 

- Radar-weather, 

- UHF-weather, 

- Sonar-propagation, 

- Layer-depth, 

- Optical  range, 

- Infrared  range. 

These  conditions  are  implemented  as  fac- 
tors which  degrade  or  enhance  the  system  per- 
formance. 

f.  Simulation_of_Targets 

A realistic  simulation  of  naval  warfare 
requires  more  participants/targets  than  just 
14;  therefore, the  number  of  cubicles/vehicles/ 
targets  was  extended  by  an  additional  112  "syn 
thetic"  targets.  Like  all  cubicles,  these 
targets  can,  at  discretion,  be  defined  as  sub- 
marines, surface  vessels,  or  aircraft,  and, 
supplementarily,  they  may  serve  as  missiles, 
torpedoes,  chaff,  or  false  targets  due  to  jam- 
ming. Targets  presently  not  needed  are  on 


standby  in  a so-called  "pool"  and  may  be  used 
when  required  by  the  scenario;  30  of  these 
targets  with  respect  to  the  particular  vehi- 
cle definition  have  a limited  number  of 

Sensors  - Radar  and  Sonar  -,  and 

Effectors  - Missiles,  Guns,  Torpedoes, 
Jammers,  and  Chaff  - available. 

The  targets  are  controlled  by  the  in- 
structors, the  cubicles  or  by  a special  pro- 
gram. 

2.  The  Training  Assistance  Program 

This  program  serves  as  a supporting 
program  for  automated  tactical  exercises  to 
control  the  flow  of  events.  This  program  may 
assume  the  following  tasks: 

- To  call  the  attention  of  the  instruc- 
tor at  the  control  console  to  observe  events 
or  their  development  thus  relieving  him  of 
routine  workload. 

- To  control  selected  exercise  parts  by 
pre-defined  criteria  which,  for  example,  will 
permit  computer  control  of  individual  targets 
or  an  entire  party. 

- To  enable  simultaneous  conduct  of 
several  minor  exercises  separate  from  each 
other. 

The  following  figure  shows  as  a summary 
the  general  layout  of  the  ASTT  concerning  the 
hard-and  software. 


Figure  5.  General  Layout  of  ASTT 
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III.  VALUE  OF  THE  ASTT  FOR  TACTICAL  TRAINING 


The  substantially  improved  capabilities 
of  the  Singer-ASTT  in  comparison  with  the  Redi- 
fon-ASTT,  and  consequently  the  value  of 
this  modern  simulator  for  the  tactical  train- 
ing are  best  expressed  by  to  the  following 
criteria. 

A.  REALISM 

Each  vehicle  type,  even  different  ver- 
sions of  one  class,  can  be  defined  individu- 
ally. Steaming  characteristics,  sensor  and 
weapon  employment  largely  reflect  actual  con- 
ditions. 

The  danger  of  obtaining  too  good  infor- 
mation from  the  computer  was  averted  by  re- 
ducing factors. 

Generally,  all  technical  possibilities 
that  might  influence  the  tactical  posture  are 
simulated  in  a realistic  way.  Thus,  artifici- 
alities in  training  are  minimized,  the  results 
are  realistic. 

The  large  number  of  targets  provides  the 
possibility  to  simulate  each  aspect  of  a 
threat  in  all  three  dimensions. 

B.  VERSATILITY 

The  trainer  offers  a wide  spectrum  of 
applicability  for  meeting  various  training  re- 
quirements. Simulation  is  possible  at  all  tac- 
tical levels,  from  simple  tactical  procedural 
exercises  through  complicated  tactical  situ- 
ations to  war  games  including  the  operational 
level  MHQ. 

Exercises  may  be  conducted  as  one-,  two- 
or  multiple-party  games.  Parties  may  be  posed 
by  cubicles,  that  is  by  students,  or  by  "syn- 
thetic" targets  which  are  either  manually-  or 
program-controlled. 

The  free  programmability  and  optional 
data  selection  almost  guarantee  that  the 
trainer  can  be  adapted  to  technological  and 
tactical  developments  and  thus  remain  up  to 
date. 


C.  EFFECT  IV ITY 

The  trainer  ensures  an  effective  and 
especially  economic  training. 

In  contrast  to  live  exercises,  simu- 
lated exercise  sequences  permit  creation  of  a 
magnitude  of  decision  situations  within  a 
relatively  brief  period  of  time. 

Mistakes  are  permitted.  The  effects  of  a 
wrong  tactical  decision  can  be  exercised  in 
the  trainer.  Situations  can  be  repeated, 
specific  actions  can  be  initiated  to  test  re- 
actions. This  will  aid  in  evoking  decision 
readiness  and  self-confidence. 


Situations  that  cannot  actually  be 
established  for  reasons  of  costs  or  security, 
such  as  weapon  employment  and  weapon  effect, 
can  be  simulated  at  the  ASTT.  Precisely,  it  is 
the  inclusion  of  weapon  effect  into  the  simu- 
lation which  conspicuously  shows  tactically 
correct  behaviour  or  wrong  decisions;  this 
aspect  of  training  can  be  covered  by  nothing 
better  than  by  a simulated  scenario. 

Ihe  trainer  is  capable  of  simultaneous 
control  of  several  minor  exercises  separate 
from  each  other,  which  permits,  for  example, 
separation  of  portions  from  a major  exercise 
and  their  cubicle-wise  or  group-wise  drill  in 
advance  of  the  actual  exercise. 

D.  TRANSPARENCY 

Program  control  of  functions  minimizes 
inadvertent  intervention  by  the  instructor  for 
example,  as  a result  of  different  reactions  in 
an  identical  situation.  On  the  other  hand,  all 
functions  can,  at  any  time,  be  monitored  by 
the  instructor  and  controlled  by  his  interven- 
tion. Pertinent  information  required  is  avail- 
able to  the  controlling  staff.  If  necessary, 
an  exercise  can  be  discontinued  and  resumed  at 
an  earlier  moment. 

E.  SIMPLICITY  OF  OPERATION 

The  multitude  of  functions  and  capabi- 
lities inherent  in  the  ASTT  requires  an  ex- 
tensive operation  section.  By  use  of  pro- 
grammed familiarization  - simultaneously 
conducted  in  all  cubicles  -,  it  is  possible  to 
acquaint  students  with  operating  the  trainer 
in  its  functions  within  a relatively  short 
time.  This  task  is  facilitated  by  a homogene- 
ous, centralized  operating  concept.  In  order 
to  fully  exploit  all  capabilities  inherent  in 
the  trainer,  an  assistant  is  assigned  to  each 
cubicle  to  support  the  relevant  command  team, 
especially  by  operating  the  commander's 
alphanumerical  display  console. 

F.  REPLAY  OF  EXERCISES 

The  debriefing  as  a very  important  com- 
ponent of  an  exercise  will  be  supplemented  and 
shaped  by  the  replay  of  the  respective  simulat- 
ed operation  presented  on  the  "Large  Screen 
Display."  Each  optional  exercise  time  - forward 
or  backward  - can  be  called  up;  the  complete 
scenario  or  section  can  be  displayed  with  a 
scale  varying  from  25  x 25  up  to  1000  x 1000 
nautical  miles.  The  flow  of  exercise  events 
may  be  in  a time  compression  mode, if  required. 

The  student  also  benefits  by  these  ca- 
pabilities; he  can  consummate  the  essential 
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sections  of  the  exercise  afterwards,  and  he 
becomes  aware  of  tactically  clear-sighted  or 
thoughtless  and  ill-advised  decisions.  The  de- 
briefing as  the  dialogue  between  the  instruc- 
tors and  the  students  is  condensed  to  the  dis- 
cussion of  tactical  facts  of  the  operation. 

IV.  VALUE  OF  THE  ASTT  FOR  TACTICAL 
DEVELOPMENT 

The  Singer-ASTT  offers  the  potential  to 
accomplish  tasks  which  were  unfeasible  with 
the  Redifon-ASTT;  these  tasks  had  been  aspects 
of  naval  warfare  which  otherwise  would  have 
required  considerable  expenditures  in  costs, 
time  and  personnel.  Artificialities  which  are 
imposed  by  safety-regulations  and  civilian 
traffic  would  detoriate  most  results  concern- 
ing naval  tactics  without  the  possibility  of 
replays.  It  is  well  known  that  general  tacti- 
cal solutions  can  never  be  derived  from  one 
life-exercises  only.  The  following  applica- 
tions grant  the  value  of  the  simulator  for  the 
complexity  of  tactical  development: 

- Preparation  and  replay  of  tactical 
exercises.  The  solutions  found  may  be  varied 
by  repetition  with  modified  parameters,  for 
example,  in  the  initial  situation  or  in  deci- 
sion situations.  This  would  permit  the  selec- 
tion of  an  optimum  solution. 

- Evaluation  of  tactical  exercises  and 
comparative  evaluation  of  diversified  solu- 
tions. Exercise  sequence  may  be  tape-recorded 
and  replayed  when  required. 

- Development  of  employment  concepts  for 
new  weapon  systems.  By  simple  parameter  modi- 
fication or  supplementation,  any  desired 
vehicle  can  be  simulated.  Such  a simulation 
with  its  account  for  reality  allows  qualitat- 

i ve  conclusions. 

- Extensive  tests  of  projected  new 
tactics  or  weapon  systems  in  the  simulator 
prior  to  contract  award. 

- Conduct  of  tactical  simulations  with 
operations  research  methods,  supported  by  the 
extensive  computer  system. 

V.  IMPACT  ON  THE  UTILIZATION  OF  THE  ASTT 
AND  ITS  LIMITATIONS 

The  multiplicity  of  the  functions  and 
the  potential  of  the  utilization  of  the  Sin- 
ger-ASTT  as  well  as  the  extensive  hard-  and 
software  presented  a considerable  impact  on 
the  utilization,  and,  therefore,  required  a 
new  concept  for  its  operation: 

- Special  knowledge  in  the  fields  of 
programming  and  computer  techniques  is  indis- 
pensable. 

- An  extensive  data  file  must  be  estab- 
lished and  continuously  complemented,. 


- To  operate  the  system,  it  is  impera- 
tive for  operating  personnel  to  acquire  spe- 
cial knowledge  and  go  through  relatively  long 
famil iarization  times. 

- For  preparation  of  exercises,  tactical 
/operational  requirements  are  to  be  converted 
into  system-specific  requirements. 

- Programs  for  control  of  specific  exer- 
cise cycles  are  to  be  defined  and  developed. 

- There  will  hardly  be  any  idle  times 
for  the  trainer,  since,  in  addition  to  tacti- 
cal exercises,  maintentance  and  programming 
also  call  for  system  operation. 

The  limitations  of  the  ASTT  primarily 
result  from  the  simulation  itself.  All  simula- 
tions have  their  constraints  inherent  in  the 
definition  of  the  problem  to  be  studied/ana- 
lysed, and  in  the  artificiality  of  the  conduct 
of  the  simulation  which  has  its  origin  in 
technological  facts. 

There  must  be  a compromise  between  the 
results  which  the  naval  officer  wants  to 
obtain  by  a simulation,  and  the  extent  of 
programming  which  the  programmer  can  offer  in 
the  frame  of  the  availabe  soft-  and  hardware; 
a simulation  sometimes  only  approaches 
real ity. 

VI.  SUMMARY 

The  Singer-ASTT  of  the  NAVTACTNGGP  is  a 
versatile  and  modern  simulator  which  renders 
an  effective  and  realistic  training  in  naval 
tactics.  Beyond  that,  it  provides  the  potential 
to  analyse  the  aforementioned  interdependence 
of  technology  and  tactics,  and  to  accept  the 
challenge  to  develop  new  tactics  vital  in 
modern  naval  warfare. 

The  excellent  practical  experience  made 
with  the  utilization  of  the  Singer-ASTT  and 
other  simulators/procedures  trainers  in  the 
German  Navy  initiated  ideas  about  a “Simula- 
tion concept  for  the  combat  readiness  training 
for  the  fleet"  under  effectiveness  considera- 
tions with  respect  to  costs,  personnel  and 
time  such  that  the  actual  training  at  sea  is 
cogently  improved. 

This  concept  will  investigate  the  pos- 
sibilities concerning  the  support  of  com- 
bat readiness  training  by  simulators  in  the 
field  of  independent  ship-squadron  training 
in  port  and  at  sea  and  formulate  appropriate 
requirements. 

These  ideas  call  for  simulators  concern- 
ing all  future  weapon  systems;  for  existing 
weapon  systems,  it  shows  possibilities  to  cre- 
ate simulator  capacities  for  the  period  1980- 
1990.  For  special  training  aspects  new  simula- 
tors, at  least  a complement  of  existing  simu- 
lators are  required  concerning 

* surface  warfare  procedures, 

* optical/optronic  director  system, 

* ESM/ECM. 
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SIMULATORS  AND  PARTS  CONTROL 


DONALD  K.  SWANSON 
Defense  Electronics  Supply  Center 


INTRODUCTION 


Parts  control  Is  a term  used  for  sever- 
al years  in  the  military  departments  to  de- 
scribe an  effort  to  control  the  use  of  non- 
standard parts  in  newly  designed  equipment 
and  weapon  systems.  Today  it  is  much  more 
than  this.  It  is  a concentrated  effort  by 
the  military  equipment  and  systems  acquisi- 
tion offices,  utilizing  Military  Parts  Con- 
trol Advisory  Groups  (MPCAGs)  to  not  only 
maximize  the  quantity  of  standard  parts  used 
in  all  new  design,  but  also  to  minimize  the 
variety  of  part  types  used. 

The  benefits  from  this  effort  are: 

a.  Increased  reliability  and  maintain- 
ability of  equipment  and  systems. 

b.  Reduced  costs  to  the  government  and 
the  taxpayer  of  part  documentation  and  quali- 
fication testing. 

c.  Fewer  new  part  types  entering  the 
DoD  logistic  inventories  and,  therefore,  re- 
duced costs  of  maintaining  those  inventories. 

This  paper  will  attempt  to  discuss: 

a.  How  the  parts  control  program  evolv- 
ed through  the  use  of  MPCAGs. 

b.  The  application  of  the  program  to 
the  simulators  developed  by  Naval  Training 
Equipment  Center  (NTEC)  and 

c.  Recent  changes  in  the  DoD  Parts  Con- 
trol System. 

EVOLVEMENT  OF  PARTS  CONTROL  PROGRAM 

Today's  expanded  DoD  Parts  Control  System  has 
evolved  over  a period  of  more  than  20  years. 
As  far  back  as  1957  the  Advisory  Group  on  the 
Reliability  of  Electronic  Equipment  reported 
that  electronic  parts  were  a major  factor  in 
field  failures  of  equipment  and  systems.  A 
1960  report  by  this  group  recommended  that 
part  specifications  be  updated  to  include 
measurable  reliability  requirements, 
thereby .making  them  more  useable  for  design. 
Thus,  the  Established  Reliability  (ER)  speci- 
fications were  developed. 

The  recommendations  of  this  report  and 
the  development  of  the  ER  specifications 
pointed  out  the  fact  that  standardization 
must  occur  during  the  design  phase  if  it  is 


to  improve  equipment  quality  and  reliability. 
Many  other  studies  by  committees  and  groups 
both  in  and  out  of  government  including  a 
special  DoD  Task  Group  have  arrived  at  the 
same  conclusion.  Therefore,  it  was  to  be 
expected  in  1971  when  the  Assistant  Secre- 
tary of  Defense  for  Installations  and  Logis- 
tics directed  that  the  military  departments 
adopt  the  parts  control  program  recommended 
by  the  DoD  task  group  for  the  procurement  of 
electronic  equipment. 

The  Defense  Electronics  Supply  Center 
(DESC)  in  Dayton  Ohio,  a supply  center  of 
the  Defense  Logistic  Agency  (DLA),  and  spe- 
cifically DESC's  Directorate  of  Engineering 
Standardization  (DESC-E)  was  selected  to 
provide  engineering  recommendations  on  part 
selection  to  the  military  procuring  activi- 
ties and  their  equipment  contractors  as  a 
key  part  of  the  overall  parts  control  effort. 
DESC-E  was  a natural  selection  as  the  first 
Military  Parts  Control  Advisory  Group  (MPCAG) 
since  fully  25%  of  all  new  national  stock 
numbers  (NSNs)  being  assigned  each  year  were 
in  the  electronic  parts  federal  suDply  classes 
(FSCs)  being  managed  by  DLA  at  DESC.  This 
meant  that  a high-degree  of  parts  prolifera- 
tion existed  in  the  electronic  parts  FSCs. 
Further,  DESC-E  already  had  a professional 
workforce  to  do  the  job.  Electronic  parts 
engineers  and  technicians  in  DESC-E  had  been 
preparing  over  90%  of  the  military  specifica- 
tions and  standards  for  the  military  depart- 
ments since  1962.  In  addition,  DESC-E  has 
also  been  acting  as  agent  for  the  military 
departments  in  managing  the  qualification 
program  and  issuing  Qualified  Products  Lists 
(QPLs)  associated  with  the  DESC  prepared 
specifications. 

By  the  time  parts  control  was  included 
in  electronic  equipment  acquisition  in  1971 
two  major  parts  control  documents  were  in 
use  - MIL-STD-891  (USAF)  Contractor  Parts 
Control  and  Standardization  Program  and  MIL- 
STD-749  Preparation  and  Submission  of  Data 
for  Approval  of  Nonstandard  Parts.  The  Air 
Force  had,  through  the  use  of  MIL-STD-891 
(USAF)  and  an  agreement  between  DLA  and  Air 
Force,  utilized  the  technical  support  of  DESC 
in  part  selection  on  a number  of  new  design 
contracts  including  the  F-111  aircraft  pro- 
gram. The  Navy  and  Army  usually  applied 
variations  of  MIL-STD-749  or  their  own  com- 
mand oriented  parts  control  methods  and 
documentation  to  the  acquisition  of  new 
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design  equipment.  Neither  Navy  or  Army 
equipment  procuring  activities  had  used  the 
services  of  DESC-E  engineers  in  electronic 
parts  selection  prior  to  1971. 

PARTS  CONTROL  IN  SIMULATORS  AND  TRAINING 
EQUIPMENT 

In  1971  the  Naval  Training  Equipment 
Center  recognizing  the  problems  to  be 
incurred  by  the  use  of  large  quantities  of 
nonstandard  electronic  parts,  became  the 
first  Navy  equipment  acquisition  office  to 
enter  an  agreement  with  a DLA  activity  (DESC) 
to  provide  part  selection  support  on  new 
equipment  contracts. 

The  agreement  called  for  the  DESC  MPCAG 
to  review  all  requests  for  approval  of  non- 
standard parts  which  were  to  be  submitted 
in  accordance  with  MIL-STD-749.  NTEC  pro- 
vided for  DESC-MPCAG  review  of  nonstandard 
parts  on  existing  ongoing  contracts  as  well 
as  contracts  yet  to  be  awarded.  Contract 


language  was  developed  to  provide  for  direct 
submission  of  nonstandard  parts  requests  to 
DESC  with  a response  required  back  to  the 
contractor  within  15  days.  An  appeal  of  the 
DESC  recommendation  could  be  made  to  NTEC 
by  the  contractor  if  he  did  not  agree  with 
the  DESC  recomnendation.  In  the  absence  of 
an  appeal,  the  DESC  recommendation  was  con- 
sidered an  NTEC  decision.  Figure  1 shows 
the  work  flow  used  by  NTEC  and  DESC  since 
1972.  The  agreement  between  NTEC  and  DESC 
is  one  of  the  strongest  parts  control  agree- 
ments yet  written  because  it  includes  the 
basics  for  a successful  program.  That  is: 

(1)  Timely,  accurate,  parts  recommendations 
made  by  up-to-date  parts  specialists  at  DESC, 
and  (2)  A strong  commitment  to  more  reliable 
better  quality  equipment  at  reduced  life 
cycle  costs  by  NTEC  program  engineers. 


Figure  1.  Parts  Control  Work  Flow  On  Simulator  And 
Training  Equipment  Contracts 
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If  we  examine  the  quantifiable  benefits 
of  a parts  control  program,  we  develop  cer- 
tain cost  avoidances  due  to  the  elimination 
of  the  use  of  nonstandard  parts.  That  is, 
we  establish  what  equipment  "could  have  cost" 
without  a strong  parts  control  effort. 

These  cost  avoidances  are  based  on  ac- 
quisition as  well  as  life  cycle  cost  consider- 
ations as  follows:  (1)  The  replacement  of  a 
nonstandard  electronic  part  type  with  a stand- 
ard part  eliminates  the  need  for  the  govern- 
ment procuring  activity  to  procure  Original 
Equipment  Manufacturer's  (OEM)  drawings  from 
their  contractors.  These  drawings  can  cost 
from  $500  to  $8,000  each  depending  on  the 
electronic  part  type  involved  according  to 
military  procuring  activities.  (2)  The  re- 
placement of  nonstandard  parts  also  eliminates 
the  need,  and  subsequent  cost  to  the  govern- 
ment, for  contractors  to  run  lengthy  and 
costly  qualifica.ion  tests  on  the  nonstandard 
parts.  ($ince  standard  parts  which  are  al- 
ready qualified  and  listed  on  QPLs  are  recom- 
mended by  the  MPCAG.)  These  tests  can  run  as 
high  as  $25,000  per  test  on  some  types  of 
integrated  circuits.  (3)  Life  cycle  cost 
impacts  of  the  replacement  of  nonstandard 
parts  include  the  prevention  of  new  nonstan- 
dard parts  from  entering  the  DoD  inventory  and 
becoming  stock  numbered  which  can  cost  over 
$1,800  per  nonstandard  part  based  on  a 10  year 
inventory  life.  (4)  Each  nonstandard  part 
replacement  also  avoids  maintenance  costs  of 
over  $3,000  over  an  estimated  10-year  life  of 
the  equipment  according  to  military  equip- 
ment managers.  This  is  based  on  fewer  fail- 
ures of  standard  parts  and  thus  less  mainte- 
nance actions. 

The  DESC  MPCAG  has  provided  parts  con- 
trol support  to  NTEC  on  63  contracts  for  sim- 
ulators and  training  equipment  since  1972. 
Thirty-three  of  these  contracts  are  still 
active  today.  DESC  engineers  have  evaluated 
a total  of  6,119  nonstandard  part  types  since 
1972  and  have  recommended  the  replacement  of 
2,291  nonstandard  parts  with  standard  Darts 
that  perform  the  same  function  in  the  equip- 
ment. Applying  the  cost  avoidance  criteria 
discussed  in  the  previous  paragraph,  we  see 
the  positive  result  of  the  NTEC  parts  control 
program  in  Table  1. 

A review  of  the  conservative  figures  in 
Table  1 shows  that  costs  in  excess  of  $21 
million  have  been  avoided  by  NTEC  and  DESC 
over  the  past  six  years  and  that  for  each 
dollar  expended  by  the  DESC  MPCAG  and  NTEC 
contractors  on  the  parts  control  program  an 
expenditure  of  $53  was  avoided. 


A review  of  a current  NTEC  contract  for 
the  Infantry  Remote  Target  System  (IRETS) 
shows  that  replacing  a relatively  small  quan- 
tity of  nonstandard  parts  replacements  can 
result  in  large  cost  avoidances  for  NTEC  and 
the  government  (see  Table  2). 

This  data  indicates  that  more  nonstan- 
dard microcircuits  were  replaced  than  all 
other  part  types  combined.  Further,  because 
of  their  complexity,  the  cost  of  OEM  draw- 
ings and  qualification  testing  of  nonstan- 
dard microcircui ts  made  the  value  of  standard 
microcircuits  (column  4)  much  greater  than 
other  types  of  devices.  The  IRETS  is  an  on- 
going contract  and  additional  cost  avoid- 
ances are  expected.  It  serves  well  here  to 
testify  to  the  continuing  success  of  the 
NTEC/MPCAG  parts  control  program  for  new 
design  simulators  and  training  equipment. 

THE  DoD  PARTS  CONTROL  SYSTEM 

Within  the  past  two  years,  major  steps 
have  been  taken  by  DoD  to  formalize  the 
overall  parts  control  effort  not  only  on 
simulators  but  on  all  DoD  systems  and  equip- 
ment development  programs.  One  step  was  the 
issuance  of  DoD  Instruction  4120.19  DoD 
Parts  Control  System  on  16  December  1976. 

The  significance  of  this  instruction  lies  in 
the  fact  that  it  makes  mandatory  throughout 
DoD  the  use  of  a parts  control  program  in 
the  development  of  major  systems  and  equip- 
ments. The  instruction  further  assigns  re- 
sponsibility for  its  implementation  through 
the  use  of  DLA  established  MPCAGs  with  the 
Military  Departments  retaining  final  author- 
ity on  part  selection  for  their  new  design 
equipment. 

Meantime  DLA  has  also  taken  steps  to 
expand  the  program  by  establishing  a second 
MPCAG  function  in  1975  at  Defense  Industrial 
Supply  Center  (DISC)  in  Philadelphia  to 
assist  in  the  control  of  mechanical  part 
types  such  as  fasteners  and  bearings  which 
are  the  second  fastest  growing  part  catego- 
ries in  the  DoD  inventory.  Recently,  two 
other  DLA  centers,  Defense  Construction 
Supply  Center  (DCSC)  and  Defense  General 
Supply  Center  (DGSC),  were  directed  to  estab- 
lish MPCAG  functions  to  provide  parts  con- 
trol support  when  requested  on  the  part  cate- 
gories managed  by  those  centers. 
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TABLE  1.  IMPACT  OF  NTEC/DESC  PARTS  CONTROL  SINCE  1972 


NOTES: 

1 / To  be  conservative,  we  assume  that  drawings  are  prevented  in  50%  of 
the  nonstandard  part  replacements. 

2 J We  also  assume  testing  is  prevented  25%  of  the  time  to  be  conservative. 

3 / Facts  show  that  each  drawing  covers  an  average  of  three  parts  and  each 
of  these  parts  is  assigned  a separate  NSN. 


TABLE  2. 


FSC 

Description 

Nonstandard 
Types  Replaced 

Value  of  j/ 
Standard  Part 

Life  Cycle 

Cost  Avoidance 

5905 

Resistors 

7 

$7,470.50 

$52,293.50 

5910 

Capacitors 

2 

7,470.50 

14,941.00 

5925 

Circuit  Breakers 

1 

4,743.00 

4,743.00 

5930 

Swi tches 

4 

7,993.00 

31,972.00 

5935 

Connectors 

6 

7,868.00 

47,208.00 

5940 

Terminals,  Lugs 

9 

4,743.00 

42,687.00 

5961 

Semiconductors 

10 

8,872.00 

88,720.0 0 

5962 

Microcircuits 

124 

12,932.00 

1,603,568.00 

5970 

Insulators 

1 

4,743.00 

4,743.00 

6145 

Wire  & Cable 

7 

7,868.00 

55,076.00 

Mi  sc. 

1 

2,555.50 

2,555.50 

172 

$1,948,507.00 

\_/  The  value  of  a standard  in  each  part  category  or  FSC  has  been  calculated 
using  the  DoD  accepted  cost-benefit  analysis  techniques  for  parts  control 
developed  by  DESC.  Further  information  on  this  subject  may  be  obtained  from 
Mr.  Charles  Gastineau,  OESC-ES,  Telephone  513-296-5086  or  Autovon  850-5086. 


The  next  step  by  DoD  was  taken  in  April 
1977  with  the  publication  of  MIL-STD-965 
Parts  Control  Program.  This  standard,  for 
the  first  time  established  coordinated  DoD 
procedures  and  requirements  for  implementing 
the  parts  control  program.  MIL-5TD-965  also 
identifies  the  Federal  Supply  Classes  (FSCs) 
for  which  parts  control  support  is  to  be  pro- 
vided. The  new  standard  supersedes  MIL-STD- 
891  (USAF)  and  MIL-STD-749  among  other  docu- 
ments and  should  be  referenced  in  future  con- 
tracts according  to  the  following  criteria: 

a.  All  major  weapons  systems. 

b.  End  items  of  equipment  where  pro- 
visioning and  follow-on  logistic  support  is 
required. 

c.  Any  other  contract  or  internal  gov- 
ernment program  where  life  cycle  cost  bene- 
fits through  parts  control  can  be  derived. 

The  requirements  of  MIL-STD-965  are 
tailorable  to  the  needs  of  the  individual 
development  program.  It  includes  two  basic 
parts  control  procedures.  Procedure  I in- 
cludes the  requirement  for  the  development 
of  a Program  Parts  Selection  List  (PPSL) 
tailored  to  the  needs  of  each  contract.  The 
PPSL  is  to  contain  a list  of  part  types  from 
which  designers  may  choose  discrete  parts 
for  use  in  their  design.  Only  those  parts 
listed  on  the  PPSL  may  be  used  in  the  design 
and  all  part  types  to  be  added  to  the  PPSL 
must  be  submitted  to  the  MPCAG  for  evaluation 
prior  to  listing.  Procedure  I is  intended 
for  use  on  most  development  contracts  where 
only  one  prime  contractor  and  few  if  any 
subcontractors  are  involved.  Procedure  II 


requires  a PPSL  but  also  includes  the  estab- 
lishment of  a Parts  Control  Board  (PCB)  to 
be  chaired  by  a representative  of  the  prime 
contractor.  It  is  intended  for  major  devel- 
opment efforts  when  numerous  subcontractors 
may  be  involved.  The  PCB  membership  typi- 
cally includes  major  subcontractors  and  rep- 
resentatives from  the  military  program  office 
at  least  one  MPCAG  representative.  The 
MPCAG  representative  is  usually  one  from  the 
MPCAG  having  predominate  interest  in  the 
parts  used  in  the  equipment.  A PCB  may  meet 
on  a monthly  basis  during  the  early  phases 
of  design  and  less  often  thereafter.  These 
PCB  meetings  provide  for  discussion  of  parts 
proposed  for  addition  to  the  PPSL  as  well  as 
an  interchange  on  critical  part  applications 
and  long  lead  time  impacts. 

Another  important  cost-effective  feature 
of  MIL-STD-965  is  that  it  provides  for  tele- 
phonic requests  for  the  addition  of  parts  to 
the  PPSL.  The  contractor's  designer  is 
urged  to  call  the  cognizant  parts  specialist 
within  the  DLA-MPCAG  and  discuss  part  selec- 
tion problems.  The  MPCAG  engineer  will  docu- 
ment the  request  and  provide  a verbal  recom- 
mendation within  two  working  days  if  not 
immediately.  Follow-up  paper  work  to  docu- 
ment the  recommendation  is  then  forwarded  by 
the  MPCAG  to  the  contractor  and  the  military 
procuring  activity.  This  obviously  saves  on 
contractor  generated  paperwork  and  results 
in  a cost  avoidance  for  the  government.  Fig- 
ure 2 shows  the  services  and  type  of  support 
typically  provided  by  DLA  MPCAGs  under  MIL- 
STD-965. 
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MPCAG  continuing  support 


MILITARY  PROCURING 
ACTIVITY 


PROGRAM  PART  SELECTION 
NON-STANDARD  PART  REQUESTS 
DRAWING  EVALUATION  REQUESTS 
PART  EVALUATIONS 


( COPY  Of 


RECOMMENDATIONS ) 

DRAWING  EVALUATIONS  (COPY  OP  RECOMMENDATIONS) 
MIL  SPECS  (new  a revised) 

QPL  INFORMATION 
SPEC  INTERPRETATION 


PART  RECOMMENDATIONS 

• DRAWING  RECOMMENDATIONS 

• PART  AV AIL AMILITY  DATA 


• PARTS  CONTROL  MEETING 
PARTICIPATION 


Figure  2.  Operation  Of  Military  Parts  Control  Advisory  Group 
(MPCAG)  In  Design  Selection 


THE  FUTURE  LOOKS  GOOD 

The  DoD  Parts  Control  Program  has  come 
a long  way  since  its  beginning  in  the  late 
1960's  and  early  1970's.  One  of  the  reasons 
for  its  success  lies  in  the  foresight  and 
positive  attitude  of  NTEC  engineers  who  came 
to  grips  with  the  problem  of  controlling  non- 
standard parts  in  their  equipment  early  in 


the  program  and  developed  solutions  through 
the  use  of  DESC's  MPCAG. 

The  total  program  has  continued  to  grow  as 
has  that  portion  of  concern  to  NTEC  and  simu- 
lator contractors.  Table  3 shows  the  growth 
of  the  program  at  DESC  through  fiscal  year 
1977. 


TABLE  3. 

GROWTH  OF  PARTS 

CONTROL  PROGRAM 

Fiscal 

Year 

Total 

Contracts 

Supported 

Life  Cycle 

Cost  Avoidance 
$ Millions 

MPCAG 

Cost 

$ Millions 

Benefit/Cost 

1973 

57 

$40 

$ .67 

58:1 

1974 

97 

53 

.77 

68:1 

1975 

142 

84 

.83 

101:1 

1976 

184 

114 

.84 

136:1 

197T 

+1977 

290 

127 

1.35 

94:1 
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It  is  expected  that  during  FY  1978  the 
DLA-MPCAGs  will  provide  parts  control  support 
to  over  330  contracts,  and  by  1980  from  350 
to  400  contracts  are  expected  to  require 
parts  control  support  from  the  MPCAGs.  The 
Parts  Control  System  is  recognized  as  an 


extremely  cost-effective  program  at  all 
levels  within  DoD.  It  is  a program  that  re- 
sults not  only  in  life  cycle  cost  avoidances 
in  equipment  acquisition  and  usage  but  also 
contributes  to  greater  reliability  in  systems 
for  the  military  services. 
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A COMPARISON  OF  COMPUTER-AIDED  TRAINING  VERSUS  CONVENTIONAL  METHODS 


DR.  RONALD  E.  OFFENSTEIN,  MR.  G.  DONALD  LODIN,  AND  DR.  GLENN  D.  BUELL 


Autonetics  Marine  Systems  Division 
Electronic  Systems  Group 
Anaheim,  California  92803 


The  capability  of  the  digital  computer 
has  been  examined  by  training  equipment  de- 
signers for  several  years  with  the  hope  of 
improving  the  speed  and/or  quality  of  instruc- 
tion. The  general  term,  computer-aided 
instruction  (CAI),  has  emerged  to  describe 
the  use  of  a computer  to  prompt,  provide 
feedback,  adjust  task  difficulty,  or  enable 
flexible  sequencing  for  learners  and/or 
training  facilities.  The  use  of  a computer 
in  these  roles  is  generally  assumed  to 
improve  instruction,  and  possibly  reduce 
recurring  costs  by  lowering  instructional 
personnel  requirements.  However,  research 
comparing  traditional  instruction  with  CAI  is 
somewhat  limited  in  terms  of  generalization 
making  verification  of  this  assumption  for  a 
particular  training  situation  difficult. 

The  study  reported  here  is  intended  to  ex- 
tend understanding  of  the  use  of  CAI  to  the 
depth  control  task  of  a remotely  controlled 
submersible  vehicle  (RCSV). 

Of  the  many  potential  computer  appli- 
cations to  the  problem  of  learning  to  control 
depth  of  a RCSV,  two  were  selected.  First, 
an  adaptive  algorithm  was  constructed  to 
adjust  the  level  of  difficulty  that  an  oper- 
ator experienced  while  learning  the  control 
task.  Second,  the  level  of  disturbance  was 
shown  numerically  to  the  operator,  thereby 
providing  feedback  about  his  performance. 

Thus,  the  computer  was  used  to  vary  the  task 
difficulty  in  accordance  with  an  operator's 
performance  and  provide  feedback  to  the 
operator  about  his  performance. 

Two  approaches  to  varying  task  diffi- 
culty were  considered  at  the  outset  of  this 
study  although  only  one  was  tested.  The 
first  approach  considered  would  have  oper- 
ated on  the  vehicle  or  control  system  dynamics 
to  lower  the  difficulty  of  control  for  be- 
ginners. An  algorithm  modifying  the  task 
difficulty  by  this  technique  would  be  based 
on  a response  variable,  and  should  be 
distinguished  from  stimulus  variable  modi- 
fications. In  contrast,  this  latter 
technique  adjusts  task  difficulty  by  varying 
a disturbing  force. 


A vehicle  such  as  a RCSV  is  designed  to  be 
stable  unless  disturbed  by  a force;  some 
disturbing  forces  arise  from  an  operator's 
control  commands  while  others  resu.lt  from 
changes  in  water  temperature,  salinity,  or 
current.  Experienced  control  operators 
minimize  the  amount  of  self-induced  distur- 
bance but  beginners  often  inject  a good  deal 
of  disturbance  which,  when  mixed  with  environ- 
mental disturbance,  is  experienced  as  an 
overwhelming  control  task.  The  use  of  an 
adaptive  model  to  add  or  subtract  environ- 
mental disturbance  in  accordance  with  an 
operator's  performance  may  aid  beginners  in 
distinguishing  their  own  disturbance  from 
that  of  environmental  sources  and,  therefore, 
enable  them  to  learn  more  clearly  the  effects 
of  their  control  commands.  This  approach  to 
an  adaptive  model  was  selected  for  the  study 
reported  here. 

In  brief,  this  study  examined  the  effects 
of  CAI  on  a beginner's  performance  in  learn- 
ing to  control  RCSV  depth.  Two  groups  of 
subjects  were  involved;  one  group  attempted 
to  learn  the  control  task  with  the  CAI  model 
while  the  other  group  had  the  same  amount  of 
practice  but  under  constantly  difficult 
conditions.  The  performance  of  the  groups 
was  compared  for  training  and  transfer  trials 
in  order  to  assess  the  merit  of  CAI  as  applied 
to  this  particular  task. 

METHOD 

Subjects 

From  a pool  consisting  of  clerical  and 
engineering  personnel,  18  volunteer  test  sub- 
jects were  selected  on  the  basis  of  avail- 
ability during  the  data  collection  period; 
the  group  was  composed  of  three  females  and 
15  males. 

Some  of  the  test  subjects  were  experi- 
enced in  the  control  of  aircraft  and/or  ships; 
this  experience  varied  considerably  from  one 
hour  in  a simulator  to  several  years  as  senior 
operational  personnel.  Fifteen  of  the  sub- 
jects had  no  prior  experience  directly  related 
to  the  experimental  task. 
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Experimental  Setting 

A modular,  reconf igurable  control  station 
was  used  as  the  operator's  station.  This 
station,  shown  in  Figure  1,  presented  stern 
plane  position,  depth  rate,  ship's  angle, 
depth  error,  speed,  depth,  and  problem  diffi- 
culty level  to  the  subject  (other  displays 
were  present  but  inactive  during  the  experi- 
ment as  they  did  not  pertain  to  the  task  of 
depth  control).  The  subject  controlled  the 
stern  planes  with  a hand  lever  that  when 
pushed  forward  caused  the  planes  to  go  into 
a dive  position  and  when  pulled  back  caused 
the  planes  to  rise. 

The  control  station  was  interfaced  with  a 
digital  computer  which  was  programmed  with  the 
equations  of  motion  for  the  simulated  vehicle. 
This  computer  also  contained  the  adaptive 
algorithm,  data  recording  software,  distur- 
bance model,  and  additional  programs  needed 
for  the  real-time  simulation.  A test  con- 
ductor's station,  shown  in  Figure  2,  was  used 
to  control  the  simulation  and  data  recording. 
The  controlled  vehicle  was  a miniature, 
remotely  controlled  submersible  approximately 
30-feet  long  with  only  stern  planes  for  pitch/ 
depth  control . 

Adaptive  Algorithm  and  Disturbance  Model 

Ten  levels  of  difficulty  were  selectable 
by  the  adaptive  algorithm  or  the  test  con- 
ductor. A subject  using  the  CAI  condition 
experienced  an  increase  in  difficulty  every 
90  seconds  providing  depth  error  did  not 
exceed  10  feet  at  any  time  during  the  90  sec- 
onds. If  a depth  excursion  larger  than  10 
feet  occurred,  the  difficulty  was  decreased 
one  level;  if  control  was  regained  and  main- 
tained for  30  seconds,  the  difficulty  was 
increased  to  the  prior  level.  Successive 
depth  excursions  greater  than  10  feet  resulted 
in  one  level  decrements  until  the  subject  was 
back  to  the  starting  level. 

The  disturbance  model  was  constructed 
from  six  sine  waves  of  different  frequencies. 
The  frequencies  were  selected  so  that  they 
would  not  be  higher  than  the  response  of  the 
RCSV  nor  lower  than  would  permit  one  complete 
cycle  within  the  training  trial-time  period. 
The  sum  of  these  sine  waves  was  multiplied 
by  a gain  and  the  level  of  difficulty  (1-10). 
The  disturbance  force  was  applied  through  the 
center-of-gravity  of  the  RCSV  causinq  a 
vertical  acceleration  which  appeared  to  the 
operator  as  a disturbance  to  the  depth  of  the 
vehicle. 


Experimental  Design,  Procedure,  and 
Performance  Measures 

A split-plot  design  was  used  for  this 
study.  This  is  a factorial  design  with  block- 
treatment  confounding;  in  the  study  described 
herein,  the  between-block  or  nonrepeated 
measures  variable  was  the  training  technique 
(CAI  vs  Fixed  difficulty)  while  the  within 
block  or  repeated  measures  variable  was  the 
transfer  task  (three  difficulty  levels:  low, 
medium,  high;  and  a depth  changing  maneuver). 
The  complexities  and  limitations  of  this 
design  were  warranted  by  the  availability  of 
subjects  and  the  nature  of  the  experimental 
treatments;  moreover,  tests  for  homogeneity 
of  pooled  variances  indicated  the  accept- 
ability of  this  technique. 


As  indicated  by  the  above  design,  two 
basic  subject  groups  were  obtained  by  random 
sampling  from  the  volunteer  pool.  One  group, 
designated  the  CAI  group,  began  each  training 
trial  with  a difficulty  level  of  1 and  pro- 
gressed as  determined  by  the  adaptive  model 
up  to  a level  of  5.  Progress  within  each 
training  trial  was  determined  by  the  subject's 
performance;  it  was  possible  for  a subject  to 
spend  most  of  the  trial  at  level  1,  level  5, 
or  any  level  or  combination  of  levels  in 
between.  The  other  group,  designated  the 
Fixed  group,  began  each  training  trial  at  a 
difficulty  level  of  5;  this  difficulty  level 
was  not  varied  during  any  of  the  training 
trials. 


Each  group  had  three  10-minute  training  trials 
separated  by  approximately  1 minute  of  rest. 
At  the  end  of  the  three  training  trials,  both 
groups  were  given  a series  of  four  transfer 
trials.  The  first  three  of  these  trials  re- 
quired the  subject  to  maintain  a depth;  each 
of  these  three  had  a different  difficulty 
level  associated  with  it  ( 1 ow= 1 , medium=5, 
h i g h= 10).  The  fourth  transfer  trial  required 
the  subjects  to  execute  a 100-foot  depth 
change.  At  the  conclusion  of  the  fourth 
transfer  trial,  the  subject  had  comoleted  the 
experiment. 


All  subjects  received  a welcoming  and 
explanatory  letter  outlining  the  basic  goals 
of  the  study;  when  a subject  arrived  at  the 
simulation  facility,  he/she  was  taken  to  the 
test  station  and  read  the  following  instruc- 
tions: 
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Figure  1.  Operator's  Control  Station 
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WELCOME : 

You  have  been  asked  to  participate  in  a study  intended  to  investigate  the  utility  of 
various  submersible  control  training  techniques.  The  task  you  will  be  trying  to  learn 
involves  remotely  controlling  the  sternplanes  of  the  submersible  vehicle  to  maintain  or  change 
depth.  The  planes  are  controlled  by  this  (point)  control  lever;  pulling  back  on  the  lever 
causes  the  planes  to  go  up  as  shown  by  this  indicator  (point).  When  the  planes  go  up,  the 
front  of  the  submersible  will  go  up  as  shown  here  (point)  and  depth  will  decrease  as  indicated 
by  the  depth  gage  (point).  The  reverse  of  all  this  happens  when  the  control  lever  is  pushed 
forward.  In  short,  pull  the  lever  back  to  go  up,  push  the  lever  forward  to  go  down. 

In  addition  to  the  indicators  that  I have  already  shown,  the  CRT  directly  in  front  of  you 
shows  depth  error;  when  the  cross  is  within  the  box,  depth  error  is  within  an  acceptable  limit. 
Your  task  will  be  to  keep  depth  at  300  feet. 

Although  ship's  course  can  be  controlled  from  this  station,  it  does  not  pertain  to  the 
present  investigation  and  should  be  ignored.  Let  me  briefly  describe  the  remaining  indicators 
in  front  of  you.  f°oint  and  explain.) 

Do  you  have  questions? 

It  is  time  for  you  to  try  controlling  the  submersible.  To  get  started,  push  these  buttons 
when  I indicate  (depth  and  course  entry).  We  will  need  to  talk  over  this  head  set  as  I will 
be  up  there  (point).  Let's  get  started. 


If  a subject  had  exceptional  difficulty  be- 
coming oriented  to  the  task,  a modest  effort 
was  made  to  get  him/her  started;  this  was 
done  for  both  groups  and  never  exceeded  a few 
orienting  comments  within  a 2-3  minute  period. 
With  the  exception  of  these  efforts,  all  sub- 
jects were  on  their  own  to  learn  the  task  of 
depth  control . 

Performance  measures  consisted  of  RCSV 
responses,  operator  control  commands,  and  the 
task  difficulty  level  as  determined  by  the 
adaptive  model  (pertains  only  to  CAI  group 
during  the  training  trials).  A listing  of 
all  performance  measures  is  given  in  Table  1. 

TABLE  1.  PERFORMANCE  MEASURES 

1 . RMS  depth  error 

2.  Integrated  absolute  depth  error 

3.  Proportion  of  time  within  +l*foot 
depth  band 

4.  Frequency  of  sign  reversals  in  hand 
controller  position 

5.  Performance  index  (mean  quadratic 
combination  of  depth  error  and  plane 
position) 

6.  Mean  absolute  plane  rate 


7.  Percent  time  planes  saturated  (displaced 
to  maximum  position) 

8.  Mean  absolute  hand  controller  displace- 
ment. 


Training  Trials 

Both  CAI  and  FIXED  groups  were  given 
three  successive  10-minute  training  trials. 
Recalling  that  task  difficulty  for  the  CAI 
group  was  determined  by  ability  to  control 
the  RCSV  depth,  it  would  be  expected  that  the 
average  task  difficulty  would  increase  as 
training  progressed.  Figure  3 shows  that  this 
did  occur;  however,  it  is  interesting  to  note 
that  after  30  minutes  of  training  the  average 
difficulty  level  had  reached  only  3.8;  the 
FIXED  group  experienced  a constant  diffi- 
culty level  of  5 throughout  the  training 
trials. 

It  is  difficult  to  directly  compare  the 
performance  of  the  separate  groups  during 
training  because  they  were  not  working  at  the 
same  level  of  difficulty.  The  question  of 
whether  the  FIXED  group  was  learning  can  be 
answered  only  indirectly  by  examining  the 
change  in  selected  performance  measures  as 
shown  in  Table  2.  These  data  generally 
reflect  modest  performance  improvements  over 
the  three  training  trials. 
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TABLE  2.  SELECTED  PERFORMANCE  MEASURES  FOR  THE  FIXED 
GROUP  OVER  THE  THREE  TRAINING  TRIALS 


Trial 

1 

Trial 

2 

Trial  3 

mean 

SD 

mean 

SD 

mean 

SD 

RMS  depth  error 

16.05 

10.05 

12.80 

16.90 

9.90 

6.00 

Frequency  of  sign 
reversals  in  hand 
control  position 

.26 

.11 

.27 

.13 

.25 

.15 

Performance  index 

18.50 

8.00 

17.20 

9.26 

16.20 

10.50 

Average  absolute 
plane  rate 

6.78 

3.40 

6.70 

3.80 

6.60 

4.00 

% time  planes 
saturated 

9.70 

16.20 

5.80 

13.00 

8.00 

13.00 

Transfer  Trials 

Three  transfer  trials  followed  the  train- 
ing trials  for  both  groups;  to  the  extent  that 
operators  had  learned  to  control  the  depth  of 
the  RCSV  during  the  training  trials,  perfor- 
mance on  the  transfer  trials  should  have  been 
improved.  If  the  CAI  method  of  instruction 
facilitated  learning  in  comparison  with  the 
FIXED  technique,  it  would  be  expected  that 
CAI  group  performance  would  be  better  than 
that  of  the  FIXED  group. 

The  other  main  effect  examined  in  this 
study  was  the  level  of  difficulty  of  the 
transfer  task.  As  it  turned  out,  this  vari- 
able accounted  for  more  variance  and  had  a 
more  reliable  effect  than  did  the  method  of 
instruction:  Analysis  of  Variance  (ANOVA) 
summary  tables  for  RMS  depth  error,  proportion 
of  time  within  a +l-foot  depth  band,  the 
performance  index,  and  average  absolute  hand 
controller  deflection  are  shown  in  Figure  4. 

Although  the  method  of  instruction  did 
not  turn  out  to  be  a statistically  powerful 
variable,  the  general  trend  in  much  of  the 
data  indicated  that  the  assumed  benefits  of 
CAI  may  have  been  realized.  This  is  reflec- 
ted by  Figure  5 in  which  values  of  the  above 
variables  are  plotted  for  the  three  transfer 
trial  levels  of  difficulty.  In  the  cases 
shown,  the  CAI  group  seems  to  maintain  better 
RCSV  control  in  that  RMS  error  is  lower  for 
difficulty  levels  one  and  two,  a greater  pro- 
portion of  time  was  spent  in  a +l-foot  depth 
band,  and  the  performance  index  is  lower. 

This  group  also  had  slightly  less  average 


absolute  hand  controller  deflection,  a mea- 
sure indicating  that  the  task  should  have 
been  less  fatiguing.  Two  of  the  observed 
differences  between  the  CAI  and  FIXED  groups 
are  significantly  different  (Pc.10)*;  in 
most  cases  the  differences  between  the  low 
and  intermediate  difficulty  levels  are  signi- 
ficant (Pc. 05)  for  both  groups. 

The  final  transfer  trial  required  oper- 
ators of  both  groups  to  execute  a 100-foot 
depth  change  maneuver.  The  purpose  in  doing 
this  was  to  examine  the  transfer  effects  of 
both  methods  of  instruction  to  a task  not 
specifically  encountered  during  training. 

The  performance  of  both  groups  was  essen- 
tially equivalent  on  this  task.  Average 
transition  duration  for  the  CAI  group  was  100 
seconds  while  that  of  the  FIXED  group  was  106 
seconds. 


DISCUSSION 

CAI  as  a method  of  teaching  remotely 
controlled  submersible  vehicle  (RCSV)  depth 
control  was  compared  with  a FIXED  difficulty 
technique.  The  CAI  group  experienced  three 
training  trials  during  which  their  performance 
determined  the  level  of  task  difficulty;  the 
FIXED  group  had  an  equal  amount  of  training 
but  their  experience  was  always  at  a constant 
(intermediate)  difficulty.  It  was  anticipated 
that  the  CAI  group  would  learn  depth  control 
more  rapidly  because  the  amount  of  distur- 
bance to  the  depth  of  the  submarine  was  ad- 
justed in  accordance  with  their  performance, 


In  concluding  that  a reliable  difference  exists  between  the  CAI  and  FIXED  group, 

there  is  less  than  a .10  probability  of  being  incorrect. 


o 


RMS  DEPTH  ERROR 


Source 

df 

MS 

F 

1.  Between  subjects 

a.  A (type  of  training) 

1 

29.43 

.3049 

b.  Sub  w'group 

16 

96.50 

2.  Within  subjects 

a.  B (transfer  task. 

difficulty) 

2 

230.32 

6.71** 

b.  AB 

2 

56.10 

1 .63 

c.  B x sub  w-group 

32 

34.33 

Proportion  of  time  within 

+ 1 foot  of  depth 

Source 

df 

MS 

F 

1.  Between  subjects 

a . A (type  of  training) 

1 

.1116 

3.455* 

b.  Sub  w-group 

16 

.0323 

2.  With  subjects 

a.  B (transfer  task 
difficulty) 

2 

.428 

36.87** 

b.  AB 

2 

.0553 

4.77** 

c.  B x sub  w-group 

32 

.0116 

Source 

Performance 

df 

Index 

MS 

F 

1 . Between  subjects 

a.  A (type  of  training) 

1 

268.39 

3.41* 

b.  Sub  w-group 

16 

78.55 

2.  Within  subjects 
a.  B (transfer  task 
difficulty) 

2 

179.03 

26.21** 

b.  AB 

2 

19.85 

2.906 

c.  B x sub  w-group 

32 

6.83 

Average  Absolute  Hand  Controller  Deflection 

Source 

df 

MS 

F 

1.  Between  subjects 

a.  A (type  of  training) 

1 

53.78 

1.33 

b.  Sub  w-group 

16 

40.239 

2.  Within  subjects 
a.  B (transfer  task 
difficulty) 

2 

126.44 

76.71** 

b.  AB 

2 

.03 

.05 

c.  B x sub  w-group 

32 

1.65 

* P <.10  **  P <.05 


Figure  4.  ANOVA  Summary  Tables  for  Four  Variables 
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PERFORMANCE  INDEX  — • RMS  DEPTH  ERROR 


thus  permitting  a clearer  separation  between 
the  effects  of  their  control  commands  (oper- 
ator induced  disturbance)  and  environmenta1 
disturbances.  Although  the  trend  in  many  of 
the  performance  measures  was  in  support  of 
this  hypothesis,  statistically  reliable  re- 
sults were  not  obtained  for  all  performance 
measures. 

An  examination  of  the  performance  of 
individuals  within  the  CAI  group  revealed 
considerable  variance.  Some  operators  never 
progressed  beyond  difficulty  level  1 during 
the  thirty  minutes  of  training  while  others 
advanced  steadily.  It  seems  reasonable  that 
this  wide  variation  in  entering  behavior  is 
primarily  responsible  for  the  lack  of  statis- 
tically strong  CAI  effects. 

Two  aspects  of  the  large  ability  differ- 
ences obtained  in  this  study  appear  to  be  of 
interest.  First,  an  appropriate  CAI  system 
must  either  be  designed  with  the  level  of 
entering  ability  in  mind  or  it  must  be  able 
to  compensate  for  these  differences.  In  the 
case  of  depth  control,  a CAI  system  would 
have  needed  to  adjust  response  variables 
(i.e.,  RCSV  dynamics)  in  order  to  lower  the 
difficulty  to  an  appropriate  level  for  the 
operators  participating  in  this  study. 

Second,  statistical  reliability  could  most 


likely  be  obtained  in  a study  like  the  one 
reported  here  by  increasing  the  number  of 
subjects;  in  effect,  increasing  the  number 
of  subjects  tends  to  diminish  the  importance 
of  the  few  operators  who  appear  to  be  unable 
to  establish  control  over  the  RCSV. 

However,  this  approach  is  not  economically 
practical  in  an  industrial  research  setting. 
Instead,  experimental  designs  that  permit  a 
maximum  of  understanding  with  a minimum  of 
subjects  and  time  are  dictated  by  the  costs 
of  test  personnel  and  facilities. 


CAI  is  most  likely  going  to  be  an  in- 
creasingly prominent  part  of  modern  instruc- 
tional systems.  Its  success  depends  upon  a 
thorough  understanding  of  the  many  variables 
involved  in  a particular  instructional  set- 
ting, making  generalizations  difficult.  In 
the  case  of  RCSV  depth  control,  this  study  has 
shown  that  the  range  of  entering  ability  must 
either  be  constrained  or  the  CAI  system  must 
be  capable  of  a broader  range  of  difficulty 
levels.  More  extensive  training  time  may 
improve  the  picture  of  CAI  presented  in  this 
study;  but  an  approach  that  fails  to  reduce 
entry  level  complexity  would  not  reduce 
the  learner's  intense  frustration  result- 
ing from  very  limited  performance  improve- 
ments during  the  initial  training  periods. 
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SUP PORTABILITY  DEMONSTRATION  FOR  FLIGHT  SIMULATORS 
CAPTAIN  LEE  0.  PUCKETT 

Acquisition  Logistics  Operations  Division/Simuiator  Program  Office 
Wright-Patterson  Air  Force  Base 


Supportablllty  covers  many  areas  in  the 
flight  simulator  world,  from  reliability, 
availability,  maintainability,  to  training, 
provisioning,  logistics  support,  or  technical 
data.  Many  of  these  areas  are  directly 
testable;  many  other  areas  can  only  be 
evaluated  by  indirect  observation.  Much 
effort  has  been  expended  in  devising  new 
methods  of  ascertaining  whether  flight 
simulators  have  met  the  needs  of  the  Govern- 
ment. A major  challenge  in  simulator 
acquisition  today  is  specifying,  designing, 
and  enforcing  support  characteristics  as 
design  parameters.  This  paper  explores  a 
contractual  method  of  obtaining  a reliable 
and  easily  maintainable  flight  simulator  for 
the  life  cycle  of  the  device. 

There  are  two  objectives  associated  with 
supportability  testing.  The  primary  object- 
ive is  to  acquire  a system  with  a high  degree 
of  correlation  between  the  specified  design 
parameters  and  the  actual  operational 
environment  and  requirements.  Simulators 
may,  in  fact,  be  one  of  the  few  areas 
wherein  such  a high  correlation  can  be 
achieved.  The  second  objective  is  to 
provide  for  a coherent  transition  period 
from  system  design  to  a final  organic 
support  capability.  Such  a transition 
needs  to  have  minimal  perturbation  on  the 
normal  support  development  process. 

There  are  many  alternatives  to  obtain 
Integrated  logistics  support  during  the 
acquisition  of  a flight  simulator.  Interim 
Contractor  Support  (ICS)  is  frequently  used 
to  obtain  the  support  necessary.  This  can, 
and  many  times  does,  simply  delay  many  areas 
of  support  due  to  the  division  of 
responsibility  between  Implementing  and 
support  commands.  In  the  past,  ICS  has 
posed  funding  problems  and  acceptance  by 
the  support  commands.  Another  approach 
could  be  to  procure  a flight  simulator, 
test  the  device  for  specification  require- 
ments, run  a reliability  and  maintainability 
demonstration,  then  transfer  the  device  to 
the  using  command.  This  typical 
acquisition  approach  leaves  the  support 
contnand  with  little  or  no  time  to  develop  an 
organic  capability  to  support  the  flight 
simulator  frequently  necessitating  costly 
extraordinary  actions.  For  flight 


simulators,  there  are  six  major  activities 
that  are  occurring  simultaneously  - spares 
and  support  equipment  delivery,  operational 
test  and  evaluation  of  the  simulator, 
technical  publication  preparation  and 
verification,  maintenance  and  operator 
technician  training,  depot- level  repair 
center  buildup,  and  follow-on  provisioning 
actions.  What  is  required  is  to  tie  these 
actions  together  with  a logical  orderly 
method,  such  that  the  acquisition  process 
verifies  each  area  before  transition  to  the 
normal  organic  posture  required  by  the  Air 
Force . 

The  supportablllty  demonstration  approach 
will  use  the  first  production  flight 
simulator  as  a test  bed  to  evaluate  support 
considerations.  A 1-year  demonstration 
and  evaluation  Is  performed  on  this  device 
in  an  operational  environment.  Three 
support  parameters  are  used  to  form  the 
evaluation  of  the  demonstration.  These 
measurable  and  contractually  enforceable 
parameters  are:  Mean  Time  Between 
Maintenance  Actions  (MTBMA),  Maintenance 
Man-hours  per  Operating  Hour  (MM)/0H),  and  a 
form  of  operational  availability.  The 
MTBMA  parameter  measures  unscheduled 
maintenance,  whereas  MMH/OH  will  cover  both 
scheduled  and  unscheduled  maintenance. 
Operational  availability  will  be  the  fraction 
of  scheduled  missions  that  can  be  flown 
without  a simulator  failure  or  some  variant 
of  this.  The  combination  of  these  three 
support  parameters,  while  not  all  encompass- 
ing, Is  sufficient  to  successfully  bind 
both  support  costs  and  training  availability. 
The  parameters  are  easily  understandable 
goals  for  those  personnel  outside  the 
acquisition  arena. 

The  supportablllty  verification, 
demonstration,  and  evaluation  concept  passes 
through  three  distinct,  though  not 
necessarily  separate  phases.  The  first 
phase  includes  the  definition  of  the 
appropriate  design  parameters  for  the 
procurement  solicitation.  This  definition 
and  the  design  innovation  It  inspires  will 
mold  the  reliability,  maintainability,  and 
availability  into  the  flight  simulator. 

Phase  II  Includes  those  actions  by  the 
contractor  and  procuring  agency  up  to  the 
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actual  on-site  start  of  demo.  The  in-plant 
actions  shall  include  qualification  testing 
using  normal  physical  configuration  audits, 
normal  system  acceptance  test  procedures,  and 
a short-term  high-risk  reliability  test.  The 
short-term  reliability  test  serves  as  an 
integrated  and  final  checkout  of  hardware 
and  software,  assuring  the  procuring  agency 
that  the  flight  simulator  has  a reasonable 
chance  of  meeting  the  more  rigorous  require- 
ments of  the  demo.  Phase  III  is  the 
demo  phase. 

After  the  simulator  passes  an  initial  check- 
out on-site,  phase  III  begins.  The  contract- 
or will  use  his  own  spares  and  approved 
support  equipment.  However,  he  may  use 
Government  Furnished  Equipment  (GFE)  on  an  as- 
available  basis,  though  not  relieving  him  of 
this  responsibility  for  providing  necessary 
equipment.  Qualification  testing  for  all 
peculiar  support  equipment  will  be 
completed  at  the  beginning  of  the  S_demo.  A 
conditional  acceptance  by  the  Government 
marks  the  beginning  of  the  1-year  demo. 

The  Government  will  continue  the  provision- 
ing, depot-level  repair  center  buildup,  and 
those  ancillary  actions  necessary  to  place 
ourselves  in  a full -support  posture  at  the 
end  of  S.  demo.  At  the  midway  point  in  the 
£ demo,  the  Government  will  conduct  a 
technical  publication  verification  of  all 
contractor  written  manuals  plus  any  support- 
ing vendors'  manuals  that  are  deliverable 
under  the  simulator  contract.  The 
verification  will  be  conducted  by  the 
trained  personnel  who  completed  the  contract- 
or school.  The  contractor  will  supplement 
the  verification  with  technical  writers  and 
system  engineers.  After  the  verification,  a 
final  set  of  technical  publications  is 
presented  to  the  Government  and  will  reflect 
the  actual  as-built  configuration.  At  the 
end  of  the  S demo,  the  contractor's  effort 
and  documentation  is  evaluated  by  the 
procuring  agency  and,  when  acceptable,  the 
contractor  receives  the  liquidation  of 
payments  with  appropriate  adjustments.  A 
final  material  inspection  and  receiving 
report  is  signed  by  the  Government  which  does 
two  things  - one,  it  allows  the  contractor 
to  receive  payment  for  the  S demo,  and  two, 
shows  the  Government  that  we  received  the 
flight  simulator  which  has  shown  to  be 
reliable  and  capable  of  meeting  the  Govern- 
ment's operational  availability  needs.  It 
should  be  noted  that  this  form  should  not  be 
signed  with  unresolved  deficiencies  or 
engineering  changes  that  affect  the  three 
support  parameters. 

To  be  effective,  numerous  details  must  be 
clearly  discussed  and  included  in  the 
original  competitive  hardware  solicitation. 

The  1-year  supportability  period  is 
written  in  the  contract  in  such  a manner  as 


to  provide  a sufficiently  large  incentive/ 
penalty  provision  to  ensure  proper  management 
and  design  attention.  The  contractor 
requirements  spelled  out  in  the  contract 
include:  providing  all  necessary  spare  and 
support  equipment  for  the  1-year  support- 
ability  demonstration  (^  demo) , technical 
publications  complete  and  validated  prior  to 
the  start  of  demo,  technical  schooling  for 
the  operators  and  maintenance  personnel  to 
include  hands-on  simulator  experience, 
providing  for  total  maintenance  responsibili- 
ty of  the  simulator  including  depot-level 
repair  capabilities,  delivering  adequate 
documentation  for  evaluation  of  the  three 
support  parameters,  and  last,  but  most 
importantly,  a clear  requirement  that  if  the 
support  parameters  are  not  met,  the  contract- 
or will  provide  the  necessary  design  changes 
and  additional  demonstration  and  evaluation 
time  at  no  cost  to  the  Government  to  clearly 
meet  the  support  parameters.  The  require- 
ments by  the  Government  will  include: 
facility  and  base  suoport  needs,  simulator 
site  observance  of  contractor  activities, 
flying  regularly  scheduled  operational 
missions  just  as  if  the  Government  owned  and 
operated  the  flight  simulator,  a formal 
verification  of  the  technical  publications 
using  contractor  schooled  maintenance 
personnel,  and  if  deemed  necessary,  perform 
a maintainability  demonstration  at  the  end  of 
the  S demo  (this  could  occur  if  the  total 
number  of  maintenance  actions  did  not  provide 
a 90%  confidence  level  estimate  of  MTBMA  and 
MMH/OH) . 

There  are  direct  and  indirect  advantages  to 
the  supportability  verification,  demonstra- 
tion, and  evaluation  concept.  Direct 
advantages  include  acquisition  of  a flight 
simulator  whose  basic  support  characteristics 
are  defined,  designed,  and  tested  in  accord- 
ance with  actual  field-level  training 
conditions,  hands-on  training  for  operators 
and  maintenance  technicians,  thereby 
increasing  personnel  stability,  lead  time  for 
spares  and  support  equipment  eliminated,  and 
support  command  in  a firm  organic  support 
posture.  Indirect  advantages  include 
Government  evaluation  of  the  integrated 
logistics  support  functions  which  are 
evaluated  from  a stable  software  and  hardware 
baseline,  controversies  related  to  test 
verses  field  performance  are  eliminated  since 
controlled  field  results  are  used,  simulator 
consumption  data  has  had  a year  to  settle 
down  and  will  provide  better  data  to  the 
support  command's  logistician,  and  finally, 
the  using  command  has  had  a year  of 
simulator  use  without  the  normal  start-up  of 
logistics  delays  that  may  adversely  affect 
availability. 

The  supportability  concept  as  presented  here 
is  a multifaceted  method  of  Integrating  the 
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many  activities  that  occur  during  the 
acquisition  of  a flight  simulator.  In 
reality.  It  also  provides  a transition 
period  until  the  Government  can  obtain  a full 
support  posture.  The  basic  guidelines  for 
the  supportability  concept  must  be  tailored 


for  each  simulator  program,  but  the  key 
factor  must  be  that  a set  of  contractually 
enforceable  support  parameters  are  specify- 
ing, designing,  and  forcing  the  simulator  to 
meet  the  operational  needs  of  the  Government. 
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MOTION  SYSTEM  OF  COMBINED  CONFIGURATION 
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SUMMARY 

Currently,  the  analysis  method  of  a motion 
system  Is  dependent  upon  the  pilots'  subjec- 
tive evaluation.  This  paper  describes  a 
method  using  the  data  recorded  by  an  Instru- 
mented dummy  to  qualitatively  analyze  a com- 
bined motion  system.  The  function  of  the 
duniny  Is  to  record  the  time  histories  of 
linear  (force)  and  angular  (moment)  accelera- 
tions Imposed  on  the  pilot's  station  during 
flight  maneuvers.  The  dummy  is  a recording 
device  Instrumented  with  three  linear  accel- 
erometers, three  gyros,  a seven- tape  cassette 
recorder  and  two  Polaroid  Instant  movie 
cameras.  The  pilot's  voice  evaluation  will 
use  one  of  the  tapes.  The  functions  of  the 
two  cameras  are  to  monitor  and  correlate  the 
flight  events. 

INTRODUCTION 

The  simulator  manufacturers  and  users 
customarily  use  the  flight  data  to  verify  the 
math  model  of  a flight  simulator  and  determine 
the  maximum  displacement  and  time  derivatives 
for  designing  a motion  system.  However, 
verifying  a motion-simulated  device,  one  Is 
dependent  upon  the  pilots'  subjective  evalua- 
tion. A motion-simulated  device  may  be  a 
motion  system,  g suit  or  g seat.  Generally, 
there  is  a lack  of  validated  data  which  Indi- 
cate a device  having  met  the  design  criteria. 
If  there  is  no  baseline  (flight)  data  avail- 
able, the  analysis  would  be  very  difficult. 

How  can  we  qualitatively  analyze  or  at  least 
interpret  the  pilots'  subjective  terms  such 
as  confused,  distracted,  irregular  and  un- 
realistic cues?  Using  the  time  histories  of 
linear  accelerations  and  Euler  angular  accel- 
erations imposed  on  the  pilot's  station  (eye- 
ball or  seat)  during  flight  maneuvers,  one 
may  correlate  the  subjective  terms  with  engi- 
neering expressions.  The  linear  acceleration 
consists  of  three  components  (A  , A , A ) 

X y i 

measured  along  the  pilot's  station  coordinates 
in  the  directions  of  X,  Y,  and  Z.  The  Euler 
angular  acceleration  consists  of  three  com- 
ponents, roll  (*),  pitch  (e),  and  yaw  (\|>) 
accelerations  measured  between  the  pilot's 
station  and  Inertia  coordinates.  The  results 
will  be  very  useful  to  the  engineers  for 
studying  and  improving  a combined  motion 
system. 

This  paper  describes  the  qualitative  anal- 
ysis method  and  the  use  of  a dummy  instru- 
mented with  sensors,  a recorder,  and  cameras 


to  record  the  flight  data  and  events  In  an 
aircraft  and  Its  simulator.  The  data  recorded 
by  the  duimy  In  an  aircraft  Is  the  baseline 
data.  The  qualitative  evaluation  of  a simu- 
lated motion  device  Is  the  analysis  of  the 
comparative  results  between  the  baseline  and 
simulator  data. 

QUALITATIVE  ANALYSIS  METHOD 

Based  on  our  experience,  the  pilots  com- 
plain against  the  low  fidelity  of  a simulator 
whenever  the  performance  data  of  the  simulator 
deviate  significantly  from  the  flight  data. 
Correcting  the  deficiencies,  one  has  to  modify 
the  performance  of  the  simulator  to  meet  the 
flight  data  and  refine  It  to  meet  the  pilots' 
subjective  evaluation.  If  the  complaint  Is 
against  the  motion-simulated  device,  the  cor- 
rection Is  solely  to  meet  the  pilots'  subjec- 
tive evaluation.  Furthermore,  flight  data  Is 
unavailable  In  a form  which  Is  suitable  for 
the  study  and  analysis  of  a motion-simulated 
device. 

The  qualitative  analysis  method  is  simi- 
lar, In  principle,  to  the  one  being  used  to 
verify  the  flight  performance  of  a simulator. 
The  differences  Include  the  collection  and 
utilization  of  the  flight  data.  The  data  of 
Interest  are  the  linear  accelerations  (Ax> 

Ay,  Az)p  and  angular  accelerations  U,  e,  d<)p 

at  the  pilot's  station.  The  vectorial  summa- 
tion of  the  linear  accelerations  will  give  us 
the  information  on  the  time  history  of  the 
phase  angle  and  the  magnitude  of  the  resultant 
vector  (force).  This  is  the  most  important 
data  required  for  providing  realistic  motion 
cues.  The  proposed  qualitative  analysis 
method  Is  quite  different  from  the  method 
currently  used  by  the  simulator  industry  in 
specifying  and  accepting  a motion  system.  The 
current  method  does  not  Include  the  use  of 
time  history  of  accelerations  of  flight 
maneuvers.  However,  the  method  does  Include 
the  use  of  the  performance  tests  in  compliance 
with  the  specified  tolerances  on  response 
frequency,  maximum  displacement  and  time 
derivatives.  Obviously,  the  current  method 
is  Inadequate  to  ascertain  the  fidelity  of  a 
motion  system  (reference  1). 

Presently,  no  single  motion -simulated  de- 
vice can  provide  the  realistic  cues  covering 
high-  and  low-dynamic  maneuvering  and  precision 
tracking.  On  the  contrary,  reference  1 cites 
a number  of  motion  systems  Including  those  of 
special  design  which  provide  negative  cues  to 
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the  pilots.  A motion  system  can  provide  on- 
set cue  (jerk,  change  of  acceleration)  and 
very  little  sustained  cue.  On  the  other  hand, 
the  g suit  and  g seat  can  provide  the  sustain- 
ed cue  but  not  the  onset  cue.  Undoubtedly, 
new  simulators  will  incorporate  a combined 
motion  system.  Hopefully,  the  analysis  method 
will  lead  us  to  better  understanding  a com- 
bined motion  system  and  to  specify  a system 
reasonably.  Figure  1 shows  the  flow  diagram 
of  qualitative  analysis  method. 

METHOD  OF  INTERPRETATION 

The  instrumented  dumny  is  the  device  to 
record  the  time  histories  of  the  flight  data 
in  an  aircraft  or  the  simulator.  The  sets  of 
data  will  consist  of  the  acceleration  data, 
correlated  indicators'  values,  visual  scene, 
and  pilot's  voice  evaluation.  Later,  three 
sets  of  data  will  indirectly  assist  the 
qualitative  evaluation.  Figure  2 illustrates 
the  time  histories  of  onset,  linear,  and 
angular  accelerations  during  a typical  take- 
off maneuvering.  Assuming  the  coupling 
effects  equal  to  zero,  the  longitudinal  (Ax) 

and  normal  (Az)  accelerations  are  only  sus- 
tained values.  For  a combined  motion  system, 
the  motion  system,  g suit  and  g seat  must 
complement  each  other  and  function  as  an 
integrated  system  to  provide  realistic  motion 
cues  to  a pilot.  After  washing  out  the  onset 
acceleration  provided  by  the  motion  system, 
the  g suit  or  g seat  must  step  in  and  function 
smoothly  and  continuously  to  provide  the  sus- 
tained acceleration  cues. 

Since  all  records  of  the  flight  data  and 
events  are  synchronous,  the  pilot's  voice 
evaluation  will  help  us  to  locate  the  negative 
cues  occurring  in  a certain  time  interval;  the 
indicators'  values  will  help  us  to  detect  the 
malfunction  of  the  simulator;  the  visual  scene 
will  help  us  to  analyse  and  correlate  the 
motion  cues.  Examining  the  curves  in  Figure 
2 has  given  us  some  clues  to  analyze  and 
interpret  the  pilot's  subjective  terms  often 
used  In  the  evaluation.  For  example,  com- 
plaint about  unrealistic  cue  will  probably 
occur  In  the  time  interval  between  tQ  and  t-| 

on  the  Ax  curve.  After  washing  out  the  onset 

acceleration,  neither  the  motion  system  nor 
g suit  could  provide  the  realistic  cues  in 
that  Interval.  The  complaint  about  uncommand- 
ed  and  confused  cues  will  probably  occur  in 
the  time  Interval  between  t^  and  t2  on  the  Az 

or  e curve.  The  cause  may  be  due  to  excessive 
time  lag  of  the  system  or  out-phase  operation. 
The  successful  determination  of  the  source  of 
the  negative  cues  Is  dependent  upon  the  avail- 
ability of  the  sets  of  baseline  and  simulator 
data  recorded  by  the  dummy.  On  the  other 
hand,  varying  the  time  lag,  magnitude  of  onset 
ar,d  sustained  acceleration  cues,  phase  angle. 
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or  gain  of  servo  control  system,  one  can 
easily  extend  the  usefulness  of  the  analysis 
method  to  study  the  pilot's  perception  of 
motion  cues. 

INSTRUMENTED  DUMMY 

Automotive  and  aerospace  industries  employ 
dimmies  for  various  experiments  such  as  car 
crash  tests  and  rocket  assisted  personnel  ejec- 
tion respectively.  In  the  task,  we  will  in- 
strument the  dummy  with  three  linear  accelero- 
meters, three  gyros  and  two  Polaroid  instant 
movie  cameras  (Figure  3).  The  accelerometers 
will  sense  the  variation  of  longitudinal, 
lateral  and  vertical  accelerations.  The  gyros 
incorporated  with  signal  processors  will  pro- 
vide the  angular-acceleration  signals.  Avoid- 
ing the  use  of  expensive  telemetry  system,  a 
seven-tape  cassette  recorder  installed  in  the 
dummy  will  record  the  time  histories  of  accel- 
erations Imposed  on  the  pilot's  station  and 
the  pilot's  voice  evaluation.  After  com- 
pleting a maneuver,  the  signals  recorded  on 
the  tapes  will  drive  a strip-chart  recorder  to 
obtain  hard  copies  of  the  records.  Driving  a 
visual  system,  the  pilot's  station  should  be 
at  the  pilot's  eyeball  while  driving  a motion- 
simulated  device,  should  be  at  the  pilot's 
seat.  To  eliminate  any  errors  due  to  the 
geometrical  location,  the  dummy  must  sit  on 
the  pilot's  seat  as  a human  pilot.  Therefore, 
a two-seat  aircraft  and  simulator  would  be 
ideal  to  carry  out  the  experiments.  Figure  4 
shows  that  the  improper  use  of  the  station 
coordinates  could  lead  to  the  computational 
errors  particularly  under  high-dynamic  maneu- 
vering. The  first  camera  located  near  the 
dummy's  eyeball  will  record  the  actual  scene 
from  an  aircraft  or  simulated  visual  scene 
from  a simulator.  The  second  camera  located 
near  the  chest  part  of  the  dummy  will  monitor 
the  indicators'  values  displayed  on  the  in- 
strument panel  of  an  aircraft  or  a simulator 
Synchronizing  the  operations  of  the  sensors, 
gyros,  recorder,  and  cameras  is  mandatory. 
Matching  the  data  with  the  events,  one  should 
regularly  mark  the  equal  time  interval  on  all 
tapes.  Otherwise,  there  is  no  way  to  analyse 
the  data. 

CONCLUSION 


The  writer  believes  that  in  addition  to 
the  current  method,  the  qualitative  analysis 
method  using  the  time  history  data  of  linear 
and  angular  accelerations  at  the  pilot's 
station  during  flight  maneuvers  will  definite- 
ly enhance  the  understanding  of  a combined 
motion  system.  One  can  use  the  results  of  the 
analysis  to  correlate  the  subjective  terms 
with  engineering  expressions  to  develop  a high 
fidelity  combined  motion  system. 
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Figure  2.  Takeoff  Maneuver 
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Figure  4.  Euler  Angles  Computation  Referred  to  Different  Coordinate  System 
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Science  - Knowledge,  often  as  opposed  to 
intuition  or  belief.  Systematized 
knowledge  derived  from  observa- 
tion, study  and  experimentation. 

Serendipity  - An  apparent  aptitude  for  making 
fortunate  discoveries  accidentally. 

Science  or  serendipity?  Versions  of  this  question  are 
too  often  asked  of  the  simulator  procurement 
process,  especially  following  the  unveiling  of  a 
multimillior.  dollar  trainer  which  incorporates  new 
and  wondrous  design  features  but  fails  to  fulfill  the 
end  user  training  requirements.  Following  a brief 
look  at  some  of  the  items  which  contribute  to  this 
criticism,  we  will  discuss  a comprehensive  approach 
to  instructional  system  management  with  emphasis 
on  simulator  procurement. 

Several  real  world  factors  may  contribute  to  any 
mismatch  of  simulator  capabilities  with  training 
requirements: 

• To  ensure  early  availability,  simulator  design  is 
often  frozen  while  tactical  equipment  design  and 
operating  procedures  are  still  evolving. 

• Insufficient  assets  are  devoted  to  front-  end 
analysis  during  the  conceptual  stages  of  instruc- 
tional program  definition. 

• Training  requirements  are  inadequately  defined 
or  are  not  reflected  in  equipment  specifications. 

• Fleet  and  school  inputs  are  requested  or  received 
too  late  to  be  incorporated  into  simulator  design. 

• The  simulator  is  developed  as  a stand-alone  entity 
rather  than  as  a component  of  an  integrated 
instructional  system. 

• Compromises  are  dictated  by  budgetary  or 
political  constraints. 

• Alternative  approaches  to  fulfilling  training 
requirements  are  not  identified  or  investigated. 

Minimization  of  these  factors  and  their  effects  is 
esseniial  to  the  timely  delivery  of  a simulator  which 
fulfills  the  maximum  percentage  of  valid  training 
requirements  within  budgetary  constraints. 

There  has  been  great  progress  in  the  past  decade  in 
developing  procedures  for  defining  that  training 
which  is  required  to  ensure  optimum  readiness  for 
both  existing  and  emerging  operational  systems.  The 


Instructional  System  Development  (ISD)  Program, 
improved  Navy  Training  Plan  (NTP)  procedures,  and 
the  establishment  of  the  Submarine  Trainer  Working 
Group  (ST WG)  are  but  a sample  of  the  myriad  efforts 
which  attempt  to  provide  early  definition  of  training 
requirements  and  the  assets  required  to  fulfill  them. 

The  Training  Information  Management  System 
(TIMS)  illustrated  in  Figure  I can  maximize  the 
effectiveness  of  such  existing  systems  and  proce- 
dures. As  presently  conceived,  TIMS  could  be  applied 
to  an  entire  instructional  system  or  any  selected 
subset  thereof,  from  tactical  equipment  conception 
through  training  system  life  cycle  support. 

The  generalized  TIMS  encompasses  all  elements 
which  could  occur  during  the  total  life  cycle  of  the 
most  complex  training  system.  As  many  as  50  or  60 
top-level  elements  could  exist.  However,  in  actual 
use,  only  those  elements  which  apply  to  a specific 
system  and  which  are  selected  by  each  manager 
would  be  actuated.  TIMS  is  not  intended  to  replace 
existing  procedures,  but  rather  to  provide  an 
organized  method  by  which  various  levels  of  manage- 
ment can  determine  and  control  all  activities 
required  during  the  acquisition  and  support  of  a total 
instructional  system.  Most  of  the  elements  shown  in 
Figure  I consist  of  activities  which  would  be 
accomplished  as  a result  of  existing  procedures.  For 
example,  many  elements  could  be  completed  during 
the  ISD  process;  TIMS  would  merely  integrate  them 
into  the  total  effort. 

TIMS  is  organized  into  four  major  functional  sub- 
systems: 

• Training  Equipment  - Includes  all  the  activities 
required  to  develop  and  acquire  both  hardware 
and  software,  from  initial  concept  studies 
through  life  cycle  configuration  management  and 
logistics  support. 

• Management  and  Control  - Consists  of  manage- 
ment activities  required  to  establish  valid 
training  requirements  and  provide  program 
direction  and  monitoring.  Includes  testing. 
Highlights  key  events  such  as  ready  for  training 
(RFT)  dates. 

• Personnel  and  Curricula  - Encompasses  activities 
which  determine  numbers  and  types  of  personnel 
required  to  operate  and  maintain  operational  and 
training  equipment.  Includes  the  development  of 
curricula  in  support  of  specific  learning  objec- 
tives. 


Figure  I.  Training  Information  Management  System  (TIMS)  Program 


• Facilities  - Includes  all  activities  required  to 
construct  a new  facility  or  modify  an  existing 
facility  to  house  all  or  part  of  the  instructional 
system. 

The  TIMS  structured  hierarchy  is  depicted  in 
Figure  2.  Both  the  top-level  program  chart  and  the 
lower  level  activity  charts  show  the  inputs,  outputs, 
processes  and  interrelationships  required  at  each 
level.  The  correspondence  between  levels  is  illus- 
trated by  lines  C and  C',  which  are  actually  the  same, 
and  which  show  the  interrelationship  between  activi- 
ties A and  B.  Individual  activity  charts  identify  all 
other  activities  from  which  an  input  is  received  or  to 
which  an  output  is  provided. 

This  hierarchical  structure  provides  to  all  levels  of 
management  easy  to  visualize  displays  of  program 
activities  at  both  the  system  and  detailed  levels. 

We  will  now  take  a more  detailed  look  at  a subset  of 
TIMS  in  which  a particular  training  situation  is 
analyzed.  Looking  back  at  Figure  I , we  note  that  the 
entire  system  is  based  upon  the  early  definition  of 
training  requirements.  This  requires  inputs  from  all 
available  sources,  including  Navy  Training  Plans, 


ISD,  personnel  studies,  and  Fleet  Project  Teams. 
These  inputs  are  integrated  and  refined  to  definitize 
the  total  system  requirements,  which  must  remain 
visible  and  be  considered  throughout  all  stages  of 
system  development.  The  training  requirements 
become  a primary  input  to  the  System  Requirements 
Specification  (SRS)  depicted  in  Figure  3. 

During  SRS  development  real  world  considerations 
are  combined  with  the  training  requirements  to 
define  the  most  effective  instructional  system  which 
can  be  achieved  within  existing  constraints.  Samples 
of  the  functions  performed  during  the  SRS  develop- 
ment are  described  below: 

• Inputs  including  the  Navy  Training  Plan,  ISD  and 
Personnel  Concept  Studies  are  used  to  determine 
the  numbers  and  kinds  of  personnel  requiring 
training,  and  the  type,  level,  length  and  location 
of  courses  to  be  provided. 

• A determination  of  the  types  and  quantities  of 
training  equipment  which  will  best  support  the 
proposed  curricula  is  made  by  consideration  of 
such  inputs  as  course  information  and  tactical 
equipment  and  human  factors  studies.  The 


222 


availability  of  existing  training  equipment  and 
cost  and  course  length  tradeoffs  are  also  con- 
sidered. It  is  during  this  activity  that  we  answer 
the  question  "Is  a simulator  the  best  means  of 
fulfilling  certain  training  requirements?" 

• Simulator  design  is  predicated  on  an  analysis  of 
training  requirements,  personnel  and  curricula 
information,  simulator  state  of  the  art  considera- 
tions and  facility  feasibility  studies.  The  design 
requirements  form  the  basis  for  the  simulator 
design  specification,  which  is  developed  in 
subsequent  TIMS  activities. 

• Facilities  concept  and  feasibility  studies  are 
performed  in  support  of  each  phase  of  SRS  devel- 
opment. Facilities  constraints  such  as  power  and 
space  availability  and  military  construction 
policies  may  impact  simulator  design.  The 
construction  design  and  performance  specifica- 
tion is  ultimately  developed  from  the  facility 
requirements. 

Our  discussion  has  illustrated  in  a limited  way  the 
methods  by  which  simulators  can  be  treated  as  an 
integral  subset  of  an  overall  instructional  system. 
Other  TIMS  features  which  counteract  the  adverse 
factors  previously  discussed  and  which  are  critical  to 
successful  simulator  procurement  can  be  seen  in 
Figure  4.  TIMS  is: 

• Flexible  - Easily  expands  or  contracts  to  meet 
specific  managerial  objectives.  Any  system 
could  be  viewed  as  a subset  of  a larger  system. 
For  example,  the  system  encompassing  the 
weapons  training  of  a new  class  ship  could  be  a 
subset  of  a higher  level  system  which  encom- 
passes the  total  instructional  system  for  the 
class.  In  contrast,  if  a new  simulator  were 
required  in  si^jport  of  existing  curricula,  only 


those  activities  specifically  related  to  simulator 
procurement  would  be  activated. 

• Graphic  - Provides  straight  forward,  real-time 
means  of  tracking  status  of  all  elements  of 
program.  Readily  supports  Management  Infor- 
mation Center  type  presentations. 

• Comprehensive  - Integrates  all  availdble  infor- 
mation into  aconcise  management  picture.  Fully 
utilizes  those  systems  and  procedures  now  in 
effect  to  define  total  training  requirements. 

• Versatile  - Can  be  computerized  or  manipulated 
by  hand,  depending  upon  the  size  and  complexity 
of  the  program.  Much  of  the  scheduling  informa- 
tion and  logistics  data  for  a combat  system 
simulator  might  best  be  mechanized,  while  this 
would  probably  not  be  cost-effective  for  smaller 
programs. 

• Responsive  - Rapidly  analyzes  program  impacts 
and  provides  alternatives  when  real  or  theoret- 
ical modifications  to  manning  or  maintenance 
philosophies,  schedules,  budgets,  home  port 
assignments  or  other  parameters  are  proposed. 

• Cost-Effective  - Provides  early  and  realistic 
definition  of  projected  budget  requirements  for 
the  life  cycle  of  the  total  system,  with  costs 
assigned  to  specific  tasks.  Allows  early  identifi- 
cation of  subsequent  budget  impacts. 

Early  implementation  of  a comprehensive  manage- 
ment system  such  as  TIMS  isessential  to  the  achieve- 
ment of  maximum  simulator  effectiveness  for  the 

dollars  available. 

Science  or  serendipity?  The  choice  is  ours. 
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Figure  4.  Expanded  TIMS  Hierarchy 
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A LOW-COST  SIMULATOR  FOR  AIR-TO-GROUND 
WEAPONS  DELIVERY  TRAINING 


MOSES  ARONSON 

Naval  Training  Equipment  Center 


SUMMARY 

A feasibility  model  of  a low-cost  air- 
to-ground  weapons  delivery  trainer  using  an 
area  of  interest  presentation  was  assembled 
and  tested  at  the  Naval  Training  Equipment 
Center.  The  primary  factor  evaluated  was  the 
pilot  acceptability  of  the  visual  cue  presen- 
tation while  performing  the  final  phase  of 
the  ground  attack  mission.  The  preliminary 
pilot  evaluation  showed  feasibility  of  the 
area  of  interest  concept  for  providing  visual 
cues  during  the  final  phase  of  the  attack 
using  rockets.  Other  attack  modes  should 
also  be  feasible.  Compared  to  other  approach- 
es developed,  the  system  cost  of  the  NAVTRA- 
EQUIPCEN  system  is  less,  and  the  risk  is  less 
to  provide  a deliverable  trainer  to  the  fleet. 

INTRODUCTION 

While  the  NAVTRAEQUIPCEN  and  the  AF 
Aeronautical  Systems  Division  have  defined 
and  studied  designs  of  air-to-ground  weapons 
delivery  trainers  since  about  1950,  the  po- 
tential users  of  the  proposed  trainers  did 
not  actually  state  their  requirements  until 
recently,  nor  did  the  upper  echelons  of  the 
OoO  look  at  these  trainers  as  a necessary 
item.  Then  the  AF  Tactical  Air  Conmand 
raised  the  question  of  an  adequate  visual 
display  for  the  A-10  flight  simulator  to 
cover  a number  of  missions  including  A/G 
attack.  In  December  1976,  the  Commander  in 
Chief,  US  Pacific  Fleet,  in  a letter  to  the 
Chief  of  Naval  Operations  and  a letter  by  the 
CNO  Deputy  Director,  Naval  Education  and 
Training,  described  the  problem  and  requested 
solutions  be  developed  as  follows:1  In 
recent  years  encroachment  and  ecological 
considerations  have  imposed  evermore  demand- 
ing constraints  on  target  use  (land  area 
target  complexes  essential  to  the  objective 
of  achieving  realistic  combat  training),  im- 
pacting adversely  on  Navy  and  Marine  Corps 
high-explosive  ordnance  delivery  practice. 
Examples  were  given  where  complaints  by  the 
local  citizenry  of  alleged  property  damage 
and  excessive  noise  impact  from  bombing 
operations  portend  continued  pressure  to 
reduce  or  desist  in  the  use  of  high-explosive 
ordnance  on  land-based  target  ranges.  Yet, 
the  retirement  for  conventional  air-to- 
ground  ordnance  delivery  training  and  ship- 
to-shore  bombardment  practice  will  continue 
for  many  years,  until  either  electronic 
scoring  techniques  preclude  the  requirement 


or  new  weapons  make  current  systems  obsolete. 
He  states  a requirement  for  an  inexpensive 
air-to-surface,  unguided  practice  bomb  that 
will  be  ecologically  nondestructive  and  noise 
suppressed.  CNO  (ONET)  added  that  other 
solutions  should  also  be  looked  at. 

The  Director,  Planning  and  Evaluation  in 
the  Office  of  the  Assistant  Secretary  of 
Defense  in  early  1976  requested  an  independent 
assessment  of  the  status  of  simulators  and 
the  role  they  should  play  in  military  tactical 
flight  training.  The  assessment  provided  by 
Calspan  was  based  on  information  obtained 
from  a literature  survey  and  survey  interviews 
with  individuals  in  industry  and  government 
who  are  knowledgeable  about  flight  simulators, 
military  tactical  flight  training  and  related 
matters. 2 One  of  the  areas  covered  was 
ground  attack. 

The  general  purpose  of  Air  Force  Project 
Number  2235  was  to  analyze  and  demonstrate 
various  technical  approaches  to  air-to-ground 
weapons  delivery  simulation  in  order  to  reduce 
the  performance  and  cost  risk  of  procuring 
aircrew  simulators  which  require  A/G  capa- 
bility.3 

An  early  device  which  had  A/G  capability, 
the  AF  Device  F-151  Gunnery  Trainer,  was 
evaluated  in  1957.  Reference  4 reported  that 
ground  target  slant  range  and  dive  angles 
were  difficult  judgments  to  make  due  to  the 
narrow  field  of  view  (15°)  and  inadequate 
resolution  (600  TV  lines).  Reduced  or  lack 
of  fidelity  required  pilots  to  approach  with- 
in 4-5,000  ft  of  the  target  to  make  a de- 
cision. The  terrain  area  presented  by  the 
target  projection  system  was  inadequate  for 
presenting  motion  perspective  cues  in  periph- 
eral vision.  The  evaluation  pilots'  estima- 
tion of  slant  range  and  dive  angle  was 
inadequate  and  consequently  they  crashed  into 
the  ground.  During  the  first  air-to-ground 
missions  In  the  training  demonstration, 
experienced  pilots  hit  the  ground  in  39  out 
of  40  passes. 

Finally,  one  more  example  of  the  need 
for  a new  approach  is  low  cost.  The  NAVTRA- 
EQUIPCEN received  a requirement  for  an  air- 
to-ground  attack  visual  attachment  for  the 
A6E  Weapons  Systems  Trainer,  Device  2F114, 
and  the  A7E  Weapons  Systems  Trainer,  Device 
2F111.  The  A6E  WST  Project  Master  Plan 
specifically  required  training  in  air-to- 
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ground  weapon  delivery  (day  mode  only).  The 
visual  system  requirement:  FOV  270°H  x 135° 
(90°  up,  45°  down),  earth/sky  projection  In 
color,  AOI  60°  (diagonal).  In  both  systems, 
the  costs  were  greater  than  the  user  was  will- 
ing to  spend,  without  the  knowledge  of  the 
exact  FOV  required.  A typical  cost  example: 
for  100  miles  of  maneuvering,  at  6500:1  scale, 
two  24'  x 72'  overlapping  model  boards  would 
be  required.  The  cost  was  estimated  at  $5M 
for  the  boards,  $3M  for  the  visual,  and 
$2-3M  for  the  R&D,  or  a total  cost  of 
$11  million. 

DESCRIPTION  OF  THE  FEASIBILITY  MODEL 

No  new  training  or  task  analysis  was 
conducted  to  establish  the  configuration  of 
the  simulator  because  the  mission  had  been 
defined  earlier. 

Reference  5 defined  the  skills  required 
for  performance  in  the  attack  mission  suc- 
cinctly. Approach  flight  skills  of  naviga- 
tion and/or  observation  while  essential 
requirements  for  a complete  mission  should 
be  well  developed  in  separate,  specialized, 
training  tasks  before  beginning  air-to- 
surface  weapons  delivery.  Of  course,  general 
flying  proficiency  should  be  adequate.  The 
training  here  must  concentrate  on  the  parti- 
cular skills  required  for  alr-to-surface 
attacks.  These  skills  include: 

a.  Attack  preparation  maneuvering  to 
emerge  on  the  weapons  delivery  flight  path 
at  a satisfactory  range  with  good  sight 
alignment 

b.  Tracking  during  the  weapon  delivery 
run 

c.  Timing  of  commence  firing  and  cease 
firing  or  of  bomb  release 

d.  Use  of  sights  and  computer  aides  to 
attack  accuracy 

e.  Integration  of  flying  with  weapons 
delivery. 

Good  judgment  In  the  planning  of  attack 
preparation  maneuvers  and  precision  in  their 
execution  are  essential  to  the  successful 
alr-to-surface  attack.  Premature  emergence 
on  the  weapon  delivery  path  exposes  the  air- 
craft to  enemy  fire  during  an  unnecessarily 
long  straight  flight  path.  On  the  other  hand, 
adequate  time  for  sight  alignment  and  track- 
ing of  target  must  be  provided.  The  more 
precise  the  maneuvering,  particularly  the 
last  roll-out  onto  the  target  track,  the 
better  the  sight  alignment  and  the  less  time 
required  for  aircraft  control  adjustment  to 
come  onto  the  firing  track.  Rapid  and 


accurate  adjustment  of  the  aircraft  to  the 
target  firing  track  and  the  development  of 
fire  control  or  bomb  release  timing  are  also 
essential.  Firing  at  excessive  ranges 
destroys  accuracy  and  timing  of  bomb  releases 
must  be  correct.  Recovery  Initiation  must  be 
properly  timed  for  continuation  of  firing 
after  recovery  initiation  is  wasteful  of 
attack  potential. 

Since  visual  cues  are  an  essential  part 
of  air-to-surface  attack  situations,  visual 
displays.  If  they  provide  the  necessary 
visual  cues,  should  be  useful  In  the  develop- 
ment and  training  of  skills  to  cope  with  such 
situations.  Ideally,  a training  exercise 
should  simulate  for  the  trainee  an  actual 
combat  situation  with  proper  targets,  full 
aircraft  maneuverability  and  range,  and 
enemy  counteraction.  He  could  then  practice 
and  develop  his  skills  In  all  phases  of  the 
problem  by  flying  air-to-surface  attack 
missions  In  the  world  of  the  visual  display. 
If  this  Ideal  situation  could  not  be  simu- 
lated, much  useful  training  and  practice 
could  be  obtained  with  a training  device 
utilizing  a visual  display  to  reproduce 
standard  training  flight  exercises  used  by 
Fleet  Training  Squadrons.  By  simulating 
these  exercises,  a visual  display  training 
device  would  provide  training  and  practice 
with  safety  and  economy  and  without  delays 
occasioned  by  bad  weather. 

Reference  6 specifically  states  the 
visual  simulation  unsolved  problems  for  the 
air-to-ground  weapons  delivery  mission  are: 
wide  field  of  view  as  the  minimum  require- 
ment and  for  full  mission  requirements,  a 
wide  field  of  view  with  medium  resolution. 

In  color. 

Reference  2 provides  a list  of  require- 
ments for  an  A/G  visual  simulation  system. 
Although  general  In  nature,  a visual  display 
must  provide: 

a.  Good  fidelity  of  the  mathematical 
models  for  the  aircraft  and  scoring 

b.  Accurate  aircraft  data,  force  and 
motion  cues 

c.  Improvements  over  present  day 
systems  In 

Field  of  View 
Resolution 
Brightness 
Gaming  Area  Size. 

Using  the  above  guidelines  and  existing 
resources  In  equipment,  a minimum  feasibility 
model  as  shown  In  Figure  1 was  designed  and 
assembled.  A description  and  the  performance 
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of  the  subsystems  follows: 

Image.  This  Is  a transparent  two-dimensional 
ground  scene,  prepared  as  described  In 
reference  6,  backlighted  by  a light  box, 
scaled  2500:1  , full  color  and  an  artist's 
rework  of  an  actual  photograph,  size  6'  x 6', 
representing  a 2.5  x 2.5  mile  area.  The 
fluorescent  light  box  provides  an  average 
Illumination  to  the  probe  of  410  FTL,  with  a 
range  of  280-710  FTL.  The  wide  range  is 
dependent  on  the  type  of  terrain  viewed  on  the 
transparency. 

Television  Camera.  Has  a l-1nch  8507A  Vldi - 
con;  Scan  Rate  1023  lines/frame,  60  Hz,  Video 
Bandwidth  32  MHz,  Automatic  Light  Range  0.1  - 
5000  fc,  and  geometric  distortion  less  than 
2 percent.  Resolves  all  ten  shades  of  gray 
on  EIA  TV  Resolution  Chart,  with  0.5  foot- 
candle  highlight  illumination  on  the  face  of 
the  camera  tube.  Resolution  is  1100  TV  lines 
horizontally  and  700  lines  vertically. 

Optical  Probe.  FOV  80°H  x 60°V,  (100° 
diagonal)  depth  of  field  4"  to  infinity,  f# 

16,  with  motion  in  pitch  +45  to  -90  degrees, 
roll  +90  degrees;  yaw  + 90  degrees,  and  zoom 
4.5:1.  Transmission  48  percent,  lens  dis- 
tortion 6 percent,  resolution  40  LP/irm  @5 
percent  MTF.  The  optical  field  of  view  was 
reduced  to  60°  diagonal  by  means  of  the  zoom 
mechanism  for  the  purpose  of  matching  the 
projector's  FOV. 

Gantry.  The  X,  Y,  and  Z travel  at  the  scale 
factor  used  for  this  simulation,  yields 

x (range,  without  zoom)  = 5.25  miles 
with  zoom  (4x)  = 23.6  miles 
(11.1  feet) 

y (lateral  travel)  = 1.9  miles  (4  feet) 

z (altitude)  = 2.1  miles  (4.5  feet) 

T-28  Cockpit.  With  inputs  to  the  computer  of 
throttle,  rudder,  elevator,  aileron,  flap  and 
wheel  signals,  and  computer  output  signals 
for  airspeed,  roll,  pitch,  R/C,  altitude, 
rate  of  turn,  slip  indicator  and  heading. 
Matches  aircraft  in  performance  and  flying 
qualities  based  on  comparison  of  calculations 
and  aircraft  data.  Servo  performance,  or 
lag  characteristics  were  not  measured,  from 
cockpit  response  through  computer  input 
through  instrument  or  visual  response. 

US  Navy  Mark  VIII  Gunslght.  As  in  the  T2C 
aircraft. 

Computer.  Analog  computer,  REAC  550,  con- 
tains mathematical  model  for  T-28  aircraft 
(reference  7)  and  the  positioning  model  to 
slave  entrance  pupil  of  optical  probe  to 
weapons  target.  (Reference  8). 


Horizon/Sky  Projector.  A point  light  source 
hair  dome,  painted  horizon  for  visual  cues 
for  pitch  of  + 90  degrees  of  travel,  roll 
+ 90  degrees  of  travel  and  yaw  of  + 180 
degrees  of  travel.  Variable  In  brTghtness 
0 to  0.5  FTL  (SG  * 1.8).  Normal  viewing  = 
0.135  FTL. 

CRT  Projector.  Scan  rate  - 1023  lines/frame, 
6()  Hz,  variable  in  aspect  ratio,  video  band- 
width 30  MHz,  with  Thomas  6M75P45  CRT  capable 
of  a line  width  of  0.0035  inches  center  res- 
olution at  a brightness  of  15,000  ft  Lamberts. 

Projection  Optics.  F.L.  4.4"  - f/1.2  + 30° 

FOV,  transmission  75  percent  minimum,  dis- 
tortion less  than  1 percent  at  1/2  FOV. 

Screen . 10'  radius,  360°  dome.  Gain  * 1.8. 
Projected  highlight  brightness  is  2.6  FTL 
while  resolution  is  800  TV  lines  horizontal 
with  10  shades  of  gray. 

The  subsystems  described  provide  a 60° 
diagonal,  hign-resolution  insetted  display 
(area  of  interest)  anywhere  in  the  pilot's 
field  of  view,  within  the  limitations  of  the 
hardware. 

The  eye  position,  and  arrangement  of  the 
CRT  projection  and  sky  projector  in  the  dome 
screen  is  illustrated  in  Figure  2. 

The  Servo  Design  Criteria  is  shown  in 
Table  I. 

Why  an  area  of  interest  display?  It  is 
assumed  for  this  design  that  there  exists  a 
center  of  interest  at  any  given  instant  of 
time  to  which  the  pilot's  attention  will  be 
devoted.  The  center  of  interest  for  the 
weapons  delivery  runs  would  be  the  target 
area  (point  of  interest)  after  crossing  the 
initial  (IP).  By  concentrating  the  TV 
system's  resolution  in  this  area  (60° 
diagonal)  about  the  pilot's  line  of  sight  to 
the  target,  current  state-of-the-art  closed 
circuit  TV  systems  can  be  used.  The  peri- 
pheral cues  of  the  horizon  and  sky  orienta- 
tion can  then  be  presented  separately. 

Figure  3 shows  a typical  "wander"  of  the 
pilot's  line  of  sight  during  the  90°  final 
turn  segment  for  a 30°  Dive  Bomb  pattern  in 
a RF4C  aircraft  from  reference  9.  This 
reference  estimates  that  a visual  display 
field  of  view  of  240°  horizontally  and  95° 
vertically  would  be  needed.  A continuous 
visual  display  of  this  size  with  the  necessary 
resolution  (800  x 5 = 4000  lines  horizontally) 
is  beyond  the  state-of-the-art. 

The  next  question  is  why  a two-dimen- 
sional model  instead  of  a more  expensive  three- 
dimensional  model?  The  various  cues  to  depth 
can  be  classified  in  terms  of  their  dependence 
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on  motion.  Many  of  the  most  compelling  cues 
such  as  interposition,  relative  size  and 
aerial  perspective  can  be  considered  essen- 
tially static,  since  they  are  present  under 
both  static  and  dynamic  conditions.  Others, 
such  as  motion  parallax  and  change  in  verti- 
cal perspective  can  only  occur  as  a result  of 
relative  movement  between  the  observer  and 
the  object  or  scene  being  viewed.  Although 
motion  parallax  is  a relatively  minor  cue  to 
depth,  it  provides  the  essential  difference 
between  imagery  derived  from  two-  or  three- 
dimensional  sources. 

A study  specifically  designed  to  investi- 
gate the  role  of  motion  parallax  in  the 
perception  of  apparent  depth  on  a dynamic  TV 
display  was  conducted  by  King  and  Fowler 
(reference  10). 

This  study  was  primarily  designed  to 
investigate  the  perceptual  process  involved 
in  viewing  target  imagery  by  means  of  a TV 
display.  It  was,  however,  considered  desir- 
able from  the  standpoint  of  potential  appli- 
cation to  use  representative  conditions  in 
terms  of  flight  trajectory,  sensor  viewing 
geometry,  and  ground  imagery.  Therefore, 
both  constant  dive  angle  and  constant  alti- 
tude approaches  were  employed  at  simulated 
velocities  consistent  with  operational  train- 
ing problems. 

The  results  of  this  study,  to  determine 
the  relative  effectiveness  of  two-  and  three- 
dimensional  image  storage  media,  indicated 
that  movement  parallax  provides  a cue  to 
depth  only  at  very  close  ranges.  It  was  con- 
cluded that  for  the  training  problem,  which 
requires  simulation  of  television  target 
imagery,  there  is  little  or  no  advantage  in 
the  more  expensive  three-dimensional  storage 
devices  for  altitudes  above  750  feet. 

Since  the  present  study  does  not  require 
simulated  operation  below  1,500  ft,  it  is  con- 
cluded that  the  two-dimensional  transparency 
being  used  is  adequate. 

The  evaluation  mission  in  the  simulator 
was  the  air-to-surface  attack  and  exercised 
the  skills  described  previously.  Reference 
11  provided  actual  flight  parameters  for  a 
rocket  delivery  in  a T-28B/D  aircraft.  It 
also  provided  the  measures  of  accuracy  needed 
to  validate  the  pilot-aircraft-sight  system 
performance. 

Before  each  flight,  the  pilot  was  briefed 
on  the  initial  conditions.  After  each  flight, 
the  pilot  was  given  feedback  as  to  his  per- 
formance. The  first  three  runs  were  for 
orientation  purposes  to  familiarize  the  sub- 
ject with  the  device  and  his  expected  perform- 
ance. After  completion  of  these  preliminary 
trials,  the  subject  was  given  a questionnaire 


to  review.  This  was  done  at  this  time  so  that 
the  subject  was  better  prepared  to  observe  par- 
ticular aspects  of  the  simulation.  This  ques- 
tionnaire was  completed  by  the  subject  when  he 
completed  all  of  his  flights. 

After  the  preliminary  trials  and  review 
of  the  questionnaire,  the  subject  made  10  con- 
secutive flights. 

Scoring  was  obtained  by  evaluating  the 
course,  airspeed,  altitude,  rate  of  dive  and 
aircraft  attitude  time  histories  and  a compar- 
ison of  the  reticle  aim  point  location  on  the 
screen  with  the  target  location  at  the  rocket 
release  point  visually  and  by  computer  readout. 
Details  are  covered  in  an  unpublished  NAVTRA- 
EQUIPCEN  in-house  Technical  Report. 

CONCLUSIONS 

On  the  basis  of  a preliminary  evaluation, 
deficiencies  in  the  simulation  hardware  can 
be  ruled  out  as  deterrents  to  the  approach 
since  they  are  engineering  changes  within  the 
state-of-the-art  and  the  consensus  of  the  five 
pilots  (one  pilot  flew  two  missions)  was  that 
the  concept  of  using  an  area  of  interest  dis- 
play superimposed  on  a wide-angle  projection 
of  the  horizon  and  sky  on  a spherical  screen 
appeared  as  a feasible  means  to  achieve  air- 
to-ground  training  in  rocket  firing  and  could 
possibly  be  extended  to  include  other  types  of 
A/G  weapons  delivery  training. 

To  verify  that  this  system  is  indeed  low 
in  cost,  a comparison  to  another  method  is 
desirable.  Perhaps  a comparison  to  the  method 
recomnended  for  the  USAF  A-10  flight  simulator 
program  for  A/G  in  reference  3 would  be  useful. 
This  is  shown  in  Table  2. 

The  cost  advantage  of  the  NAVTRAEQUIPCEN 
system  is  in  the  2D  model,  camera,  probe 
versus,  the  dedicated  computer  for  CIG  in  the 
image  generator.  For  the  display,  the  cost 
advantage  comes  from  one  CRT  projector  versus 
many  infinity  optics  windows  in  the  CIG/M0SAIC 
approach. 

From  a performance  standpoint,  the  servo, 
optical  and  photometric  performance  of  this 
system  exceeds  that  of  other  similar  projec- 
tion systems  such  as  the  LAMARS  at  the  USAF 
Flight  Dynamics  Laboratory  and  the  DMS  at 
NASA,  Langley  Research  Center. 

It  is  recomnended  that  a further  evalu- 
ation of  the  concept  should  be  attempted  with 
a larger  pilot  sample  and  with  a larger  field 
of  view  area  of  interest  prior  to  acquisition 
of  units  for  the  fleet.  This  question  of  a 
larger  A0I  field  of  view  was  very  recently 
raised  in  a new  study  on  the  ASPT  reported  in 
reference  12. 
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TABLE  1.  SERVO  DESIGN 

CRITERIA 

RANGE  TRAVEL 

MAX  VELOCITY  MAX  ACCELER- 

POSITION 

AT  ION 

RESOLUTION 

Gantry 

Long itudinal 

+3.6’ ,-7.5 ' 

20' /sec 

3. O' /sec  2 

0.0025  ' 

Lateral 

+ 2.0' 

20 '/sec 

3 .0  '/sec  2 

0.0025  ' 

Vertical 

+0.0.-4  .5" 

20 '/sec 

3. O' /sec  2 

0.0025  ' 

CRT  Projector 

Azimuth 

+ 180° 

10 Rad/sec 

50 Rad/sec2 

NA 

Elevation 

+90° 

10 Rad/sec 

50 Rad/sec2 

NA 

Horizon/Background  Projection 

Roll 

+90° 

20  °/sec 

NA 

0.01° 

Pitch 

+90° 

20° /sec 

NA 

0.01° 

Yaw 

+180° 

6 °/sec 

NA 

0.01° 

TABLE  II. 

COMPARISON  OF 

NAVTRAEQUIPCEN  A/G  SOLUTION  WITH  CIG/M0SAIC 

§2 

astern 

Method 

NAVTRAEQUIPCEN 

CIG/MOSAIC 

Image 

Generation  2D  model 

Light  Box 

Optical  Probe/Gantry 

TV  Camera 

Servo  Control 

CIG  Computer 

Image 

Display 

CRT  Projector 

Point  Light  Source 

Servo  Control 

Multi  CRT's 
with  infinity 
optics  windows 
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Figure  2.  EYE  POSITION  AND  ARRANGEMENT  WITHIN  SPHERICAL  SCREEN 


Figure  3.  30°  DIVE  BOMB  - 90°  FINAL  TURN  SEGMENT 
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THE  CFA  CONCEPT,  A NEW  APPROACH  TO  TRAINER  ENGINEERING 
CHANGES  IN  THE  FIELD 


MOSES  ARONSON 

Naval  Training  Equipment  Center 


INTRODUCTION 

The  practice  of  modification  engineering 
or  engineering  change  support  as  performed 
at  NAVTRAEQUIPCEN  is  based  on  the  premise 
that  trainer  service  life  extension  and/or 
its  conversion  to  a later  configuration  is 
an  economical  and  feasible  means  of  avoiding 
procuring  additional  trainers  and  having  to 
pay  the  full  production  costs.  This  concept 
is  similar  to  that  used  by  NAVAIRSYSCOM 
in  the  conversion  of  aircraft.  (Reference 
1).  Actually,  the  NAVTRAEQUIPCEN  trainer 
conversion  concept  is  not  new.  One  of  the 
earliest  conversions  I recall  was  that  of  the 
Device  2F16  F9F-5  OFT  to  the  Device  2F46  F-1E 
(FO-4)  OFT  in  the  late  1950s.  Some  of  the 
facets  of  modification/modernization/ 
conversion,  such  as:  what  are  these  changes?, 
what  is  the  magnitude  of  the  program?,  how 
is  it  being  done?,  and  where  is  it  going? 
will  be  covered  in  this  paper.  Let  us  look, 
at  the  kinds  of  changes  which  need  to  be 
engineered. 

The  configuration  and  performance  of  the 
training  device  in  the  field  and/or  in 
acquisition  can  be  affected  by  the  following 
types  of  changes: 

a.  Trainer  unique  hardware/software 
changes:  Those  changes  to  the  train- 
ing device  only  (trainer  hardware 
and/or  software)  which  correct  or 
improve  trainer  performance  or  modify 
trainer  capabilities  - better 
maintainability,  improved  safety,  etc. 

b.  Changes  to  the  operational  tactical 
digital  computer  program  tapes:  Those 
changes  to  the  operational  digital 
computer  programs  and  tapes  which 
affect  weapons  system  performance  in 
such  a way  that  changes  to  the  train- 
ing devices'  performance  may  be 
required. 

c.  Changes  to  the  hardware  of  the  weapons 
system  being  simulated:  These  changes 
to  the  weapons  system  hardware  (or 
conjunctive  hardware/software  changes) 
which  affect  weapon  system  performance 
in  such  a way  that  changes  to  the 
training  devices'  performance  and 
configuration  may  be  required. 

A trainer  hardware/software  change 


categorization  system  is  employed  ty 
NAVTRAEQUIPCEN  in  the  trainer  configuration 
management  process  to  define  the  steps  that 
must  be  followed  for  any  change  considera- 
tion. Proper  categorization  of  trainer 
change  requests  will  permit  certain  types  of 
hardware/software  changes  to  be  implemented 
quickly  without  undue  time  delay  or  cost, 
while  ensuring  that  changes  with  significant 
functional  or  performance  and  documentation 
impact  are  thoroughly  evaluated  prior  to 
implementation.  All  potential  changes  to  a 
training  device  may  be  assigned  any  one  of 
four  categories  as  defined  in  the  following 
paragraphs: 

a.  Category  TA  - A change  to  the  trainer 
software  which  requires  a conjunctive  change 
to  the  trainer  hardware. 

b.  Category  TB  - A change  to  the  trainer 
software  only  which  affects  (1)  the 
functional  configuration  or  performance  of 
the  trainer,  or  (2)  requires  changes  to 
requirements  or  user  documents  such  as 
trainer  operator's  manuals,  training  manuals, 
or  trainer  maintenance  manuals. 

c.  Category  TC  - All  other  trainer  soft- 
ware changes.  Category  TC  changes  are 
concerned  only  with  efficient  and  accurate 
programming  of  trainer  programs  and  may 
require  modification  of  computer  program 
design  documentation.  Examples  of  Category 
TC  changes  include  modifying  a computation 

so  that  it  is  performed  in  a fewer  number  of 
steps  or  so  that  it  requires  less  storage, 
and  correction  of  coding  errors. 

d.  Category  TD  - A change  to  trainer 
hardware  only  which  affects  (1)  functional 
configuration  or  performance  of  the  trainer, 
(2)  maintainability/reliability  of  the 
trainer,  or  (3)  training  improvements/ 
improved  capability/training  characteristics. 

A second  facet  of  the  program  is  its 
relative  magnitude.  Look  at  the  life  cycle 
cost  for  a highly  modified  air/warfare  class 
of  trainers,  the  F-4  Aircraft  Weapons  System 
Trainers.  Figure  1 shows  the  distribution 
of  trainer  costs  among  initial  acquisition, 
replenishment  of  repairable  supply  system 
stock,  operating  and  maintenance  costs  and 
finally  the  modification  costs.  The  life 
cycle  cost  covers  the  period  from  about  FY 
1957  to  FY  1982.  This  end  date  is  not  the 
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obsolescence  date  for  the  trainers  but  the 
current  limit  on  computation  of  costs.  It 
can  be  seen  that  the  Initial  acquisition 
cost  and  that  spent  on  modifications  to 
date  are  nearly  equal  (41  versus  36.6%)  and 


that  the  Integrated  logistics  support  and 
operating  costs  are  about  22%  of  the  life 
cycle  cost  to  FY  1982.  The  total  life 
cycle  cost  from  about  1956  to  1982  Is 
estimated  at  $68fc  million. 


100 


80  J 


60  A 


40 


20  A 


(15%)  0&M.N 


MODIFICATION 

APN 

(36.6%) 


PROCUREMENT 

RDT&E/APN 

(41%) 


T 


■Military  Personnel  Dedicated  to  0&M 
of  Simulators.  (2.7%)  0&M,N 

Civilian  Personnel  Dedicated  to  0&M 
of  Simulators. 

Repair  of  Repairable,  Installation; 
Log.  Sup't  Mods,  Replacement  of 
consumables.  (4.4%)  0&M,N 

Replenishment  of  Repairable  System 
Stock.  (.3%)  0PN 

Engineering  Changes  Resulting  in 
Training  Characteristics  Modifications 
(Hardware  and  Software). 


Initial  Procurement  of  Trainer  and 
Support  Items 


NOTE:  Funding  Identification  - 0&M,N:  Operation  and  Maintenance,  Navy; 
0PN:  Other  Procurement,  Navy;  APN  - Aircraft  Procurement,  Navy: 
RDT&E:  Reasearch,  Development,  Test  and  Evaluation,  Navy. 


Figure  1.  Life  Cycle  Cost  of  a Highly  Modified  Air  Warfare  Trainer 
(FYs  1957  - 1982  Period). 


238 


The  next  two  facets:  How  Is  It  being 
done  and  where  Is  It  going  will  be  covered  by 
the  balance  of  the  paper. 

THE  COGNIZANT  FIELD  ACTIVITY  (CFA)CONCEPT 

By  Department  jf  Defense  directives  there 
are  three  levels  of  maintenance  - orni  ,za- 
tional,  intermediate,  <*nd  depot.  Generally, 
depot  level  maintenance  organizations  are 
also  authorized  to  perform  modifications. 

The  term  depot  level  maintenance  would  apply 
equally  to  an  industrial  type  effort  such  as 
that  performed  by  the  original  equipment 
manufacturer  (OEM)  or  that  performed  by  an 
industrial  like  government  activity.  The 
maintenance  concept  for  training  devices 
tends  toward  two  levels  - organizational  and 
depot  level.  This  then  restricts  the  level 
of  modifications  to  two  also  - the  small 
tasks  which  do  not  require  special  equipment 
to  Install  the  changes  or  consume  more  than 
a few  days  to  install,  and  the  large  tasks 
which  require  fabrication  and  assembly  of 
components  or  changes  in  digital  computer 
programs  beyond  the  capability  of  the  user 
or  organizational  maintenance  personnel.  The 
trainer  modification  engineering  project 
engineer,  when  investigating  a proposed 
trainer  change,  must  determine  the  level  of 
resources  available  to  implement  the  change. 
Thus,  should  the  change  be  performed  under 
contract  by  an  industrial  concern/software 
house  or  by  within-the-Navy  resources? 

Pursuing  the  division  of  labor  further,  there 
is  also  the  question  of  who  has  the  engineer- 
ing capability  to  develop  the  engineering 
change  and  can  supply  the  changes  to  the 
integrated  logistic  support  items  and  to  the 
technical  documentation.  Further,  by 
directive  of  the  Chief  of  Naval  Education  and 
Training  Support  (2)  specifically  assigned 
levels  of  authority  to  perform  modification 
engineering  were  provided  for  the  efficient 
use  of  personnel  and  resources.  These  levels 
are: 

Level  1 - Those  engineering  changes  which 
may  be  performed  by  the  custodian/using 
activity  providing  that  the  change  is  less 
than  40  man-hours  (design,  debug,  installation 
and  documentation);  does  not  affect  hardware; 
does  not  affect  the  operational  computer 
programs. 

Level  2 - Any  change  which,  through 
circumstances,  cannot  be  performed  by  the 
user  as  a Level  1 change  or  which  requires 
hardware,  conjunctive  hardware/software  or 
operational  computer  software  changes  and  is 
less  than  1,000  man-hours  (design,  debug, 
installation  and  documentation). 

Level  3 - Any  change  exceeding  the  limits 
of  Level  2. 


In  conjunction  with  these  performance 
levels  are  the  assigned  levels  for  authoriz- 
ing the  changes  as  follows: 

a.  Level  1 - At  the  discretion  of  the 
CFA,  and  in  coordination  with  the 
custodian/user,  approval  authority 
may  be  assigned  to  the  user  to 
perform  tho°f  changes  defined  as 
Level  1.  V.anges  to  be  accomplished 
at  this  level  must  be  concurred  in 
by  the  CFA.  Documentation  developed 
as  a result  of  a Level  1 change  is 
to  be  provided  to  the  CFA  for  co- 
ordination with  other  device  users. 

If  coordination  dictates  no  other 
user  requirement  for  the  change,  the 
change  can  be  classified  as  "local" 
and  may  not  be  incorporated  into  the 
baseline  documentation  by  the  CFA. 
Dedicated  memory  locations  will  be 
established  by  the  CFA  and  utilized 
for  this  purpose. 

b . Level  2 - All  changes  exceeding  the 
character  and  limitations  established 
for  a Level  1 change  will  be  sub- 
mitted in  accordance  with  reference 
(6).  The  CFA  will  provide  all 
services,  material  , and  documentation 
necessary  to  perform  and  support  the 
required  changes  up  to  1,000  man-hours 
of  design,  debug  documentation  and 
installation.  This  action  will  be 
accomplished  within  the  control 
parameters  established  by 
NAVTRAEQUIPCEN.  Changes  exceeding 
1,000  man-hours  may  be  assigned  to 
the  CFA  on  an  exception  basis  and 
with  CNET  SUPPORT  concurrence. 

c.  Level  3 - Those  changes  exceeding 
the  Level  1 and  2 will  be  forwarded 
to  the  NAVTRAEQUIPCEN  for  further 
action  in  accordance  with  reference 
(6).  Decisions  regarding  further 
implementation  will  be  provided  by 
the  NAVTRAEQUIPCEN  in  conjunction 
with  the  appropriate  sponsoring 
agency. 

You  will  notice  that  this  distinction 
in  levels  of  performance  and  authority  is 
different  from  the  Naval  Air  Systems  and 
the  former  Naval  Ordnance  Systems  Command 
approach  to  delegation  of  engineering  effort 
for  service  equipment.  (References  (3)  and 
(4)).  For  these  two  Commands,  upon 
completion  of  equipment  production,  full 
engineering  responsibility  for  a specific 
weapons  system  is  transferred  to  a field 
activity;  and  the  field  activity  performs 
the  same  engineering  functions  and  technical 
coordination  that  the  Assistant  Program 
Manager  did  at  Headquarters.  (Referenced) ) . 
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Assignments  of  limited  engineering 
responsibility  of  specific  trainers  were 
started  in  1975  by  requesting  the  CNET 
SUPPORT  field  activities  to  recommend 
training  equipment  which,  because  of 
geographic  location  and/or  developed 
experience,  can  best  be  assigned  to  their 
cognizance.  The  proposed  assignments  would 
provide  efficiency  in  engineering  change 
design  effort  and  would  develop  organiza- 
tional expertise  in  particular  units  of 
training  equipment.  The  items  included 
would  be  characterized  by  a high-level  of 
complexity  and  an  expected  high-tncidence 
rate  of  engineering  changes  during  its  life 
cycle.  The  first  list  of  assignments  under 
this  concept  was  issued  in  July  1977. 

Some  definitions  of  terms  are  now  in 
order  so  that  the  concept  operation  will  be 
easier  to  follow  later  on. 

a.  Engineering  Change  Support  (ECS)  or 
Modification  Engineering.  The  engineering 
effort  necessary  to  add  to  or  alter  the 
design  of  an  equipment  in  such  a manner  or 

to  such  an  extent  as  to  change  its  operation- 
al capabilities  or  its  design  attributes  of 
performance,  reliability,  maintainability, 
safety,  operability  and  parts  interchange- 
ability  or  to  render  it  capable  of  alterna- 
tive or  additional  use.  The  resultant 
design  change  includes  baseline  maintenance 
of  support  documentation,  computer  software, 
and  support  material. 

b.  Cognizant  Field  Activity  (CFA) . The 
field  activity  of  the  Naval  Education  and 
Training  Support  Conmand  designated  to 
perform  engineering  change  in  support  of 
specifically  designated  training  equipment. 

c.  Software  Support  Activity  (SSA). 

An  organization  designated  to  perform  digi - 
tal  computer  software  support.  Unless 
specific  circumstances  require  separate 
consideration,  simulator  computer  software 
support  responsibility  Is  integral  to  the 
assignment  of  CFA  responsibility. 

d.  Computer  Software  Support. 

Engineering  change  support  as  related  to 
simulator  digital  computer  software, 
including: 

(1)  Modifications  to  computer  soft- 
ware as  necessary  to  meet  training  or 
logistic  support  requirements. 

(2)  Design  and  development  of  new 
computer  software  as  required. 

(3)  Baseline  management  of  computer 
software  involving  the  Identification, 


collection,  storage,  reproduction  and 
distribution  of  computer  programs  and 
associated  documentation  essential  to 
daily  operations. 

(4)  Computer  program  housekeeping 
including  assembly/compile  operations  and 
those  design,  development  and  modification 
functions  concerned  with  the  clarity  and 
efficiency  of  computer  programs. 

e.  Operational  Computer.  That 

portion  of  the  weaoons  system  computer  used 
in  training  equipment. 

ENGINEERING  CHANGE  SUPPORT  AT  THE  CFA  LEVEL 

At  this  point  we  will  focus  on 
the  performance  of  engineering  change 
support  which  is  to  be  performed  at  the 
CFA  level.  Figure  2 shows  the  resources 
available  to  perform  various  levels  of 
modification  and  the  relation  of  the  CFA 
to  these  levels. 

Functions  of  the  CFA.  These  functions 
are  defined  in  relation  to  the  deployment 
date  or  the  Ready- for- Training  (RFT)  date 
of  a specific  training  device.  Those 
performed  prior  to  the  RFT  date  are: 

a.  Assumes  assist  position  to 
NAVTRAEQUIPCEN  during  the  acquisition 
process.  In  this  capacity,  the  CFA  will: 

(1)  Generate  and  develop  the 
engineering  change  support  plan.  This 
effort  is  to  be  accomplished  in  coordina- 
tion with  the  development  of  the  simulator 
Life  Cycle  Logistic  Support  Plan  addressed 
by  Reference  (7).  Details  regarding  plan 
format  are  provided  in  Volume  1 of  the 
approved  outline  for  the  Life  Cycle 
Logistic  Support  Plan.  The  Engineering 
Change  Support  Plan  shall  be  submitted  to 
CNET  SUPPORT  via  the  NAVTRAEQUIPCEN. 

(2)  Provide  inputs  in 
establishing  the  composition  of  the  data 
and  training  required  to  the  NAVTRAEQUIPCEN 
to  ensure  inclusion  in  trainer  acquisition 
budget  submissions. 

(3)  Perform  reviews  of 
Training  Equipment  Change  Proposals  (TECP) 
and  to  provide  appropriate  comments. 

(4)  Attend  the  in-plant  test 
and  acceptance  of  the  equipment  and  may 
attend  design,  technical,  contract  or 
Fleet  Project  Team  conferences  and  meet- 
ings as  requested.  Assignment  of  this 
responsibility  should  not  be  construed  as 
assignment  to  a specific  division/ 
department  within  the  CFA.  It  would  be 
expected  that  group  travel  requirements 
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are  minimized  through  the  use  of  a single 
representative  to  address  all  interests 
(Field  Engineering  Representative  support, 
simulator  modifications,  etc.;  of  the  field 
activity  whenever  possible. 

(5)  Participate  in  the  configu- 
ration Management  configuration  physical 
audits. 


Those  functions  performed  after  the 
RFT  date  are: 

a.  Acts  as  the  single  activity 
for  the  accomplishment  of  Level  1 and  2 
engineering  changes  in  the  field.  The 
changes  undertaken  are  to  be  performed  in 
accordance  with  reference  (6)  and  to  the 
limits  promulgated  by  the  NAVTRAEQUIPCEN. 


Scope  of 

Engineering  Change 


USER/CUSTODIAN 

CFA 


SSA 

CFA 

NAVTRAEQUIPCEN 


CFA  * 

NAVTRAEQUIPCEN 
GOVERNMENT  DEPOT 
LEVEL  MAINTENANCE/ 
MODIFICATION  ORG. 
CONTRACTOR 


b.  Investigates  variances  In  chanae 
requirements  and  reconmends  appropriate 
solutions  including: 

(1)  An  operational  and  function- 
al description  of  the  problem  and  reconmeded 
solutions. 

(2)  A technical  approach  to  the 
problem  including  a definition  of  the 
performance  effectiveness  of  the  training 
system  with  and  without  the  change. 

(3)  Cost  and  scheduling  infor- 
mation for  implementing  the  change  by  the 
CFA. 

c.  Designs  and  develops  trainer 
engineering  changes  with  associated  revised 
documentation.  (Digital  computer  programs 
would  be  issued  as  updated  iterations  of  the 
baseline  data.) 

d.  Maintains  the  identified  base- 
line reflecting  field  originated  changes  and 
other  changes  as  assigned. 

e.  Responds  to  trouble  reports 
related  to  the  engineering  change  program. 

f.  Interfaces  with  user  activities 

in  the  coordination  of  change  requirements. 

NAVTRAEQUIPCEN  FUNCTIONS.  To  assist  the 
CFAs  in  performing  their  function  the  Naval 
Training  Equipment  Center  will  get  them 
involved  as  soon  as  practical  during  the 
equipment  conceptual  phase.  It  is  during 
this  phase  that  necessary  planning  and 
direction  will  take  place  which  will 
ultimately  lead  to  a successful  operationa 
phase  engineering  change  support  program. 
Assignment  of  a CNET  SUPPORT  Field  Activity 
as  CFA  on  a simulator  or  family  of 
simulators  is  one  which  constitutes  a 
commitment  of  the  resources/expertise  of  the 
entire  activity. 

a.  Upon  issuance  of  the  initial 
planning  documents  designating  the  firm 
requirement  for  a new  item  of  training 
equipment,  the  NAVTRAEQUIPCEN  will  recommend 
the  appropriate  field  activity  to  serve  as 
the  CFA  and  notify  the  designated  activity  by 
formal  correspondence.  The  letter  will  be 
released  via  CNET  SUPPORT. 

b.  Provide  Military  Characteristics. 
Program  Master  Plans  and  related  procurement 
documentation  to  the  CFA. 

c.  Provide  the  CFA  with  a list  of 
deliverable  data  items  under  the  terms  of  the 
training  equipment  contract  and  place  the 
CFA  on  automatic  (identification  on  DD  Form 
1423)  distribution  of  selected  data  items. 


d.  Provide  the  CFA  Information 

on  program  and  technical  conferences  and 
meetings.  * 

e.  Solicit  CFA  for  technical  data 
and  training  requirements  and  include 
requirements  in  subsequent  acquisition 
planning. 

f.  Provide  the  CFA  with  contrac- 
tor submitted  Trainer  Engineering  Change 
Proposals  (TECP)  for  comments  and 
reccmendations. 

After  the  trainer  RFT  date,  the  Naval 
Training  Equipment  Center  will: 

a.  In  accordance  with  its 

assigned  mission,  provide  overall  engineer- 
ing direction  for  the  implementation  and 
control  of  the  engineering  change  program. 

b.  As  aDprooriate,  orovide  the 

CFA  with  information  regarding  non-field 
originated  change  requirements  (AFC's, 
ORDALT's,  SHIPALT's  etc.)  for  review. 

c.  Provide  contract  administra- 
tion services  for  contractor  engineering 
support  services  to  supplement  the  CFA's 
manpower  resources. 

Responsibilities  of  the  CFAs.  As  soon  as 

a CFA  is  designated  for  a specific  training 
device  or  family  of  trainers  it  is  to  plan 
for  the  implementation  of  the  hardware/ 
software  transition  from  trainer  develop- 
ment contractors  to  in-service  activities 
through  an  orderly  and  systematic  approach. 
To  achieve  the  objective  of  the  task  it  is 
necessary  to: 

a.  Establish,  train,  and  maintain  a 
dedicated  group  of  personnel  to  provide 
engineering  change  support  services  for  the 
assigned  trainers. 

b.  Establish  a documentation  data  base 
to  support  the  engineering  change  support 
task. 

c.  Review  applicable  ECP's  and  prepare 
TECPs  in  accordance  with  MIL-STD-480. 

d.  Respond  to  field  requested 
modifications  in  accordance  with  reference 
(6). 

e.  Implement  approved  changes 
resulting  from  approved  cost  and  lead  time 
estimates. 

f.  Provide  task  management/ 
coordination  for  successful  completion 
of  the  transition. 
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The  success  of  the  transition  schedule 
hinges  heavily  on  three  factors:  technical 
competence,  satisfactory  baseline  documenta- 
tion, and  a realistic,  agreed  upon  develop- 
ment contractor  phaseout  schedule.  As  such, 
transitioning  of  training  device  support 
from  development  contractor  to  an  In-service 
facility  Is  predicated  on: 

a.  Requested  CFA  team  training  being 
approved  and  Implemented. 

b.  Required  engineering  data/documenta- 
tion being  obtained  in  a timely  manner. 

c.  Phase  out  of  trainer  development 
contractor  being  accomplished  according  to 
the  Navy  hardware/software  transition 
schedule. 

The  phaseout  of  development  contractors 
will  be  accomplished  according  to  a three- 
phase  schedule  agreed  upon  by  the  trainer 
development  contractor,  training  equipment 
sponsor,  NAVTRAEQUIPCEN,  and  the  CFA.  The 
three  phases  are: 

Phase  0 - (1)  Provide  and  assign 
personnel  to  the  designated  family  of 
trainers  and  through  a coordinated  training 
plan,  obtain  required  weapons  system  technical 
expertise,  (2)  Make  provisions  for  the 
obtainment  of  a configuration  end  item 
documentation  data  base,  (3)  Consolidate 
training  plans  into  a master  training  D?an 
to  be  forwarded  to  trainer  sponsor,  (4) 

Review  ECPs  and  modification  requests  to  be 
accomplished  under  an  in-service  effort, 

(5)  Establish  a central  trainer  documentation 
data  base  repository,  and  (6)  Coordinate  with 
development  contractors  and  trainer  sponsor 
for  orderly  and  timely  transition. 

Phase  1 - This  phase  is  a combined 
effort  involving  both  the  in-service 
activities  and  the  development  contractors. 

It  is  during  this  phase  that  both  the  in- 
service  activities  and  contractors  coordinate 
the  incorporation  of  the  current  engineering 
changes.  Upon  completion  of  these  changes, 
the  baseline  will  be  documented  and  all  data 
will  be  delivered  to  the  in-service 
activities  for  continuation  of  the  engineer- 
ing change  support  program. 

Phase  2 - In-service  Engineering 
Change  Support  Program  is  established  and 
in  operation. 

I will  now  explore  one  facet  of  this 
operation  in  more  detail. 

Recording  of  Engineering  Changes  and 
Their  Distribution.  A central  repository 
for  hardware/software  documentation  data  for 


each  trainer  Is  established  at  the 
NAVTRAEQUIPCEN.  The  central  repository 
contains  and  constitutes  the  master  files 
for  the  specific  trainer  ECS  task  and, 
as  such,  will  be  under  the  control  of  the 
NAVTRAEQUIPCEN.  The  centra  1 resposltory 
Is  designated  as  the  central  point  for 
accounting  of  all  data  for  the  assigned 
specific  training  equipment.  The  CFA  will 
have  cognizance  of  working  documentation 
and  will  forward  revised  master 
reproducibles  to  the  central  repository  for 
filing  as  they  are  generated. 

It  must  be  noted  that,  prior  to  the 
issuance  of  any  hardware  or  software  base- 
line data  to  effect  a contractor  modifica- 
tion effort,  the  CFA  will  be  consulted  by 
NAVTRAEQUIPCEN.  This  action  is  necessary 
since  the  repository  may  not  house  the 
most  up-to-date  data  due  to  in-house  on- 
going modification  efforts,  lag  time 
inherent  in  the  documentation  chain,  etc. 

Types  of  documentation  to  be  stored  in 
the  central  repository  are: 

a.  Reproducibles  of  engineering  and 
maintenance  drawings. 

b.  Reproducibles  of  maintenance 
handbooks. 

c.  Historical  engineering  hardware/ 
software  reports  such  as  design  reports. 

d.  Instructional  material  applicable 
to  on-site  and  factory  training  courses. 

e.  Documentation  required  for  support 
of  computer  programs  and  tape  update  (i.e., 
Master  Tapes,  Listings,  Computer  Hardware 
Manuals,  etc.). 

The  CFA  has  generally  identified  to 
NAVTRAEQUIPCEN  the  documentation  required 
to  support  its  hardware/software  efforts. 
Data  requirements  for  specific  trainers  can 
oe  adjusted  when  recommended  by  the  CFA 
acting  in  consonance  with  representatives 
of  the  users. 

To  ensure  the  adequacy  of  the  baseline 
documentation  and  to  ensure  its 
availability  for  an  in-service  effort,  the 
following  actions  will  be  taken  by  the 
NAVTRAEQUIPCEN: 

a.  Define  certain  software  data  per 
SECNAV  Instruction  3560.1. 

b.  Establish  a review  process/cycle 
for  both  hardware  and  software  documenta- 
tion between  the  NAVTRAEQUIPCEN  and  the 
CFA. 
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c.  Include  the  CFA  on  the  appropriate 
DO  1423s  for  automatic  receipt  of  documenta- 
tion. 

d.  Establish  a firm  schedule  for 
documentation  delivery. 

e.  Ensure  the  adequacy  of  data  procured 
by  use  of  the  CFA  review  process  for 
specific  devices. 

f.  Ensure  that  trainer  sponsor  is  aware 
of  documentation  procurement  efforts  and  the 
possible  resultant  problems  if  it  is  not 
procured  due  to  shortage  of  funds. 

The  primary  categories  of  documentation 
affected  by  software/hardware  changes  to  the 
specific  trainer  are: 

a.  Requirements  documents  such  as 
trainer  system  performance  specifications. 

b.  Design  documents  such  as  computer 
program  performance  and  design  specifications. 


detailed  program  and  subprogram  design 
documents,  design  description  drawings, 
trainer  program  listing,  manufacturing 
and  maintenance  drawings,  Criteria 
Reports. 


c.  User  documents  such  as  maintenance 
publications,  training  manuals,  lecture 
materials  and  PMS  cards. 

d.  Internal  documentation  required 
for  the  configuration  management  process. 

e.  Test  plans  and  procedures. 

Maintenance  of  trainer  system 
specifications  will  be  the  responsibility 
of  the  NAVTRAEQUIPCEN  as  provided  for  in 
MIL-STD-480.  Control  of  working  master 
design  documentation  is  the  responsibility 
of  the  CFA.  In  addition,  the  CFA  will 
identify  and  describe  the  impact  of 
trainer  hardware/software  design  changes 
on  all  documentation. 


MODIFICATION  PROGRAM  BY  DOLLAR  VOLUME  OF  TASK 
(5  YEAR  AVERAGE)  AS  OF  FY  77 
FOR  ALL  LEVELS  OF  EFFORT 


Figure  3. 
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Updating  user  documentation  such  as 
trainer  maintenance  manuals,  training 
manuals,  lecture  materials  will  be  the 
responsibility  of  the  NAVTRAQUIPCEN  who 
is  also  responsible  for  integrating  software 
design  change  information  into  this  user 
documentation.  Test  plans  and  procedures 
will  be  developed  and  updated  by  the  CFA  as 
required  to  support  applicable  design, 
integration,  verification  and  acceptance 
testing  responsibilities.  Approval  of  these 
documents  will  be  the  responsibility  of 
NAVTRAEQUIPCEN.  The  computer  programs  will 
be  revised  by  the  CFA  to  reflect  changes 
made  to  the  program.  Patches  will  not  be 
issued.  The  revised  reassembled  computer 
program  will  be  incorporated  into  the  change 
kit  for  distribution.  Revised  masters  will 
be  provided  to  NAVTRAEQUIPCEN  for  approval 
and  storage  in  the  master  repository. 

MODIFICATION  WORKLOAD 

What  is  the  magnitude  of  the  modifica- 
tion workload?  If  we  take  the  statistics 


for  a recent  year,  we  can  see  the  distri- 
bution of  effort.  Figure  2 showed  the 
performers  by  level  of  effort.  The  next 
figures  give  some  specific  directions  for 
the  effort.  Figure  3 shows  the  distribu- 
tion of  dollar  volume  of  modification 
effort.  Shown  is  a 5 year  average  through 
FV  1977  and  indicates  that  most  modifica- 
tion tasks  are  in  the  under  $100,000  value. 
The  number  in  this  dollar  value  range  is 
actually  larger  as  modifications  costing 
$5,000  or  less  for  aviation  trainers  or 
$20,000  or  less  for  surface  and  sub-surface 
trainers  may  be  lumped  together  and 
authorized  in  a group  as  a single  task. 

Another  way  of  looking  at  the  workload 
is  by  warfare  areas.  Table  1 shows  the 
numbers  of  tasks  assigned,  irrespective  of 
funding  levels,  to  the  CFAs.  The  number 
of  modification  tasks  performed  for 
aviation  trainers  greatly  exceeds  those 
assigned  for  all  the  other  warfare  . reas. 
This  is  only  natural  since  presently  air 
warfare  trainers  represent  77%  of  all 


TABLE  1.  MODIFICATION  TASKS  BY  WARFARE  AREAS  ASSIGNED  TO  CFAs  IN  FY  1978 


WARFARE  AREA 
AIR 

SURFACE 
SUBSURFACE 
LAND  (MARINE  CORPS) 


NUMBER  OF  TASKS 
218 
69 
28 
6 


TOTAL  321 


MODIFICATION  TASKS  BY  WARFARE  AREAS  ASSIGNED  TO  CFAs  IN  FY  1978 


245 


trainers  In  inventory.  While  no  discussion 
has  been  presented  of  the  Implementation  of 
trainer  digital  computer  program  changes> 
Table  2 shows  the  distribution  of  effort  in 
the  CFAs  between  hardware  and  software 
changes.  This  figure  shows  only  the  training 
characteristics  modifications  (two  of  the 
three  types  described  earlier)  but  these 
would  represent  about  90%  of  all  modifica- 
tions. It  can  be  seen  that  for  air  warfare 
trainers  hardware  changes  still  predominate 
probably  due  to  the  nearly  constant  intro- 
duction of  weapon  system  hardware  changes 
while  in  the  surface-subsurface  area  the 
software  changes  are  a slight  majority.  This 
might  be  due  to  the  refining  of  ocean 
environmental  models  or  SENSOR/TARGET 
characteristics  as  well  as  increasing  the 
capability  of  the  instructor's  station  and 
the  trainer  as  a whole.  Future  trends  in 
workload  can  be  looked  at  two  ways  by  yearly 
trends  and  product  mix.  Table  3 shows  the 


jrowth  In  the  total  modification 
engineering  program  and  the  in-house  or  CFA 
share  of  the  program.  Future  years  funding 
has  not  been  definltized  but  it  looks  like 
it  will  grow.  Note  that  the  in-house  share 
of  program  is  not  increasing  as  rapidly  as 
the  total  program.  The  other  way  to  look 
at  the  program  is  by  product  mix.  Table  4 
shows  the  present  distribution  of  training 
equipment  in  use  by  warfare  area  and  the 
projected  mix  in  Fiscal  Year  1984  based  on 
present  acquisition  trends.  You  can  see 
that  the  total  share  of  nonaviation 
devices  has  grown  from  23%  to  36.5%  of 
total  inventory.  What  does  this  mean  on 
workload?  Table  2 showed  that  the  surface 
and  subsurface  trainer  modification  was 
software  change  intensive,  thus  the 
conclusion  could  be  that  by  Fiscal  Year 
1985  or  so  the  major  CFA  labor  effort  will 
be  in  software  as  opposed  to  the  current 
hardware  intensity. 


WARFARE  AREA 


HARDWARE 


SOFTWARE 


AVIATION 


82.3  Man-Years 


51 .5  Man-Years 


SURFACE-SUBSURFACE  9.4  10.8 


TOTAL  91 .7  Man-Years  62.3  Man-Years 


TABLE  2. 

TRAINING  CHARACTERISTICS  MODIFICATIONS  ONLY: 
DISTRIBUTION  OF  EFFORT  BETWEEN  HARDWARE 
AND  SOFTWARE  (FY  78) 

ASSIGNED  TO  CFA's 


FISCAL  YEAR 


DOLLAR  VALUE  ($1000 ’s) 


IN-HOUSE  (CFA)  SHARE  ($1000's) 


1978 

$15,000  (Approximate) 

$3,760 

1977 

11,938 

3,013 

197T  (1/4  Year) 

1,638 

636 

1976 

5,475 

2,742 

1975 

4,166 

2,485 

TABLE  3. 

TOTAL  MODIFICATION  ENGINEERING  PROGRAM  TREND 
(ALL  LEVELS  OF  EFFORT) 


L 
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EQUIPMENT  IN  USE  ANTICIPATED  AGGREGATE 

FY  77  FY_84 


AIR 

77.0 % 

63.5% 

SURFACE 

15.3% 

25.4% 

SUBSURFACE 

7.7 

TABLE  4.' 

11.1% 

DISTRIBUTION  OF  IN-USE  ASSETS  BY 
WARFARE  AREA  CURRENTLY  AND  IN  FY  84 

CONCLUSION 

In  conclusion,  we  can  make  a number  of  3. 

general  statements  about  the  Modification 
Engineering  Program  and  the  CFAs.  The 
Modification  Engineering  Program  is  an 
approach  used  by  the  NAVTRAEQUIPCEN  to  extend 
the  life  of  current  training  equipment  and 
conform  them  to  the  everchanging  operational 
weapons  systems  they  provide  training  for.  On 
a specific  trainer  life  cycle  cost  comparison, 
the  modification  costs  can  nearly  equal  the  4. 

initial  acquisition  costs  but  are  spread  over 
a larger  time  frame.  Most  modification  tasks 
are  under  $100,000.  The  CFAs  provide  an 
engineering  service  to  the  NAVTRAEQUIPCEN 
during  the  acquisition  phase  and  after  RFT 
take  on  the  major  engineering  change 
responsibilities  for  small  changes.  The 
CFAs'  workload  will  increase  but  not  at  the  5. 

same  rate  as  that  of  contractor  performed 
modification  tasks.  And  finally,  the  CFAs 
expand  the  NAVTRAEQUIPCEN  in-house  engineer- 
ing labor  base.  6. 
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THE  RjLE  OF  SIMULATION  IN  LARGE-SCALE  ISD 


DR.  G.  P.  KEARSLEY  AND  DR.  A.  I O'NFAl 
Courseware 


The  scope  and  complexity  of  major 
Instructional  Systems  Development  (ISD) 
programs  and  the  sophistication  and  detailed 
definition  of  the  particular  ISD  models 
being  utilized  in  the  military  have  greatly 
increased  in  the  last  few  years.  The 
advanced  planning  and  day  to  day  management 
required  for  such  programs  has  become 
commensurately  more  difficult  and  complex. 
The  interactions  among  the  many  resources, 
personnel,  and  schedul inq  requirements 
involved  increases  the  difficulty  of 
identifying  specific  sources  of  problems 
and  responding  to  them  without  causing 
other  problems  elsewhere.  In  response  to 
this  management  challenge,  interest  is 
greatly  increasing  in  comprehensive, 
real-time  integrated  information  management 
systems  for  ISD  which  incorporate  a variety 
of  flexible  management  projection  and  sim- 
ulation capabilities.  It  is  becoming 
essential  to  give  managers  the  tools  to  test 
the  impact  of  their  decisions  before  they 
commit  themselves  to  courses  of  action 
whose  consequences  are  not  completely 
understood.  This  paper  discusses  the 
nature  and  implications  of  such  simulation 
capabilities  in  ISD  program  management  with 
particular  reference  to  two  projects 
recently  completed  under  contract  to 
Defense  Advance  Research  Projects  Agency 
(DARPA)  and  the  Naval  Training  Equipment 
Center  (NAVTRAEQUIPCEN) . 

In  the  context  of  ISD  management 
systems,  simulation  capabilities  provide 
two  major  functions:  projection  of 
existing  trends  and  modeling  of  changes  to 
the  present  system  parameters.  Thus,  it  is 
possible  to  determine  the  estimated  start/ 
finish  dates  for  any/all  tasks  or  events 
in  the  ISD  program.  It  is  possible  to  ask 
such  questions  as  "with  current  personnel 
and  resource  conditions  when  will  the 
course  be  finished?,"  "how  much  of  each 
skill  or  other  resource  available  will  be 
used?,"  or  "how  much  time  of  person  X can 
be  released  while  maintaining  the  projected 
completion  date?"  If  a planned  media 
alternative  is  written  out  for  some  reason, 
what  will  be  the  effects  on  the  program? 

For  example,  suppose  a course  being 
developed  has  40  videotapes  specified  but 
the  project  is  now  running  over  budget. 

It  would  be  possible  to  explore  the 
possibility  of  replacing  those  videotapes 
with  a more  economical  media  such  as  slide/ 
tapes  to  see  what  effect  they  will  have  on 
budget,  resource  utilization,  and  project 


Incorporate' 


completion  dates.  Another  possibilty  is  to 
furnish  the  system  with  a desired  .ompletion 
date  and  calculate  the  conditions  necessary 
to  meet  that  deadline  including  all  resource 
needs.  With  these  types  of  "what  if" 
capabilities,  it  is  possible  to  determine 
the  real  cause  of  problems  or  delays.  For 
example,  it  may  be  found  that  the  planned 
hiring  of  two  more  instructional  psycholo- 
gists will  have  no  effect  on  production 
rate  or  completion  deadline.  With  the  help 
of  some  exploration  using  the  simulation 
capabilities,  it  might  be  revealed  that  the 
real  bottleneck  is  in  review  and  editing 
personnel . 

Simulation  capabilities  such  as  those 
described  above  have  been  provided  in  two 
ISD  management  systems  developed  by 
Courseware  Inc.  The  Author  Management 
System  (AMS)  is  an  operational  prototype 
system  developed  under  contract  to  DARPA 
implemented  on  minicomputer  in  BASIC  (1). 

It  provides  a set  of  management  support 
capabilities  for  the  design,  development, 
production,  and  ongoing  curriculum  mainten- 
ance phases  of  ISD.  It  tracks  the  exact 
stage  of  completion  of  each  identified 
course  component  and  monitors  the  work- 
loads, current  assignments  and  deadlines 
of  all  personnel.  A major  built-in  feature 
of  AMS  is  the  generation  of  reports  which 
indicate  estimated  and  projected  dates  of 
completion  or  resource  expenditures.  For 
example.  Figure  1 shows  a total  course 
projection  report  indicating  estimated 
projected  resource  utilization  and  Figure  2 
illustrates  a manpower  projection  report 
which  indicates  overall  projected  project 
completion  date.  If  the  amount  of  time 
estimated  to  be  required  of  a particular 
class  of  personnel  on  some  project  activity 
is  altered,  it  is  possible  to  see  the  over- 
all effect  on  project  costs  and  completion 
date.  Thus,  AMS  allows  the  investigation 
of  the  effects  of  salary  increase,  addition 
or  loss  of  specific  personnel  or  classes  of 
personnel,  budget  cuts,  changes  in  syllabus 
or  media,  changes  in  project  procedures, 
or  changes  in  the  time  frame  available  for 
developments. 

The  Computer-Aided  Training  System 
Development  and  Management  (CATSDM)  project 
funded  by  NAVTRAEQUIPCEN  (2)  has  resulted 
in  the  specification  of  a system  of  programs 
and  databases  to  support  all  phases  of  the 
ISD  process  in  the  military  training  context. 
These  programs  are  organized  according  to 
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— — *«  TOTAL  course:  projection  — — 

USING  PRESENT  RATE  OF"  COMPLETION 
MONDAY,  MARCH  6,  1978 


UNITS  MONEY 


RESOURCE 

E5TIHATED 

PROJECTED 

ESTIHATED 

PROJECTED 

RATE 

01 

instructional  psychologist 

8S.00 

8S.00 

1714. SS 

1714.55 

1.000 

02 

SUBJECT  NATTER  EXPERT 

#53-00 

6 S3.  00 

8044.44 

8044.44 

1.000 

OS 

AUTHOR 

Sll.SO 

Sll.SO 

424. IS 

424.15 

1.000 

04 

TYPIST 

213.00 

213.00 

428.48 

428.48 

1.000 

OS 

INSTRUCTIONAL  TECHNOLOGIST 

82-00 

82-00 

1240.34 

1240.34 

1.000 

04 

EDITOR 

317.00 

317.00 

2344-12 

2344.52 

1.000 

07 

ARTIST 

444.  SO 

444.  50 

17733. 03 

13733-03 

1.000 

06 

PRODUCTION  HAN ACER 

44.  SO 

44.50 

404.42 

404.42 

1.000 

04 

WORD  PROCESSOR 

201. SO 

201- 50 

1404.1* 

1404. 10 

1.000 

10 

PASTER-UPPER 

77.  SO 

77. SO 

SO«.  40 

508.40 

1.000 

11 

HEDIA  EXPERT  <S/T> 

120.00 

120.00 

844.00 

844. 00 

1-000 

12 

SCRIPTWRITER 

310-00 

310.00 

4143.30 

4143.30 

1.000 

S3 

PHOTOGRAPHER 

100. oo 

100.00 

34  s.  00 

345.00 

1.000 

14 

HEDIA  EXPERT  <S/0) 

84.00 

84.  00 

1048-2# 

10*8.2# 

1.000 

IS 

LYRICIST 

70.00 

70.00 

241. SC 

241.50 

1.000 

14 

CQNPQSER 

70.00 

70.00 

535.  SO 

S35.S0 

1.000 

17 

CHOREOGRAPHER 

301.00 

301.00 

1349.55 

1344. SS 

1.000 

18 

DIRECTOR 

41-00 

41-00 

57#.  44 

S?4.44 

1.000 

14 

KAZOO  PLAYER 

41.00 

41.00 

34S.0S 

34S.  8S 

1.  000 

20 

SINGER 

41.00 

41.  00 

644. IS 

#46. IS 

1.000 

21 

DANCER 

41.00 

41.00 

303.03 

303.  03 

1.000 

22 

DEC 

184.00 

184.00 

4151.52 

41S1.S2 

1.000 

UMi.  91  ..SOS.  93 

ORICINAL  ESTIMATE:  MASOS. 93  DOLLARS 
PROJECTION  AT  PRESENT  RATE:  4AS0S. 93  DOLLARS 
UHICH  IS  0 DOLLARS  DIFFERENT  FROM  ESTIMATE 


Figure  1 . 
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******  MANPOWER  FOR  COURSE!  COMPLETION  ****** 
USING  PRESENT  RATE  OF  COMPLETION 
TUESOAY.  MARCH  7.  1978 


— HOURS  TO  COMPLETE  THE  PROJECT  — 


RESOURCE 

estimated 

PROJECTED 

AVAILABLE 

RATE 

01  INSTRUCTIONAL  PSYCHOLOGIST 

79.00  - 

282.82 

- 

3102.00 

3.580 

02  SUBJECT  MATTER  EXPERT 

020.00  - 

1264.80 

- 

2831.28 

2.040 

03  AUTHOR 

511. SO  - 

511.50 

- 

1942.98 

1.000 

04  TYPIST 

213.00  - 

213.00 

- 

2S09.80 

1.000 

OS  INSTRUCT TONAL  TECHNOLOGIST 

82.00  - 

82.00 

- 

2346.24 

1.000 

00  EDITOR 

317.00  - 

317.00 

- 

1646.88 

1.000 

07  ARTIST 

944.50  - 

944.50 

- 

1229.52 

1.000 

08  PRODUCTION  MANAGER 

40.50  - 

46.  SO 

- 

879.84 

1.000 

09  UQRO  PROCESSOR 

201.50  - 

201.50 

- 

817.80 

1.000 

10  PASTER-UPPER 

77.50  - 

77.  SO 

- 

710.64 

1.000 

11  MEOIA  EXPERT  (S/T) 

119.00  - 

SOS. 7S 

924.96 

4.2S0 

12  SCRIPTWRITER 

310.00  - 

310.00 

- 

902.40 

1.000 

13  PHOTOGRAPHER 

100.00  - 

100.00 

- 

1128.00 

1.000 

14  MEDIA  EXPERT  (S/0> 

80.00  - 

465.26 

- 

479.40 

5.410 

15  LYRICIST 

70.00  - 

70.00 

- 

676.80 

1.000 

10  COMPOSER 

70.00  - 

70.00 

- 

22S.60 

1.000 

17  CHOREOGRAPHER 

301.00  - 

301.00 

- 

338.40 

1.000 

18  OIRECTOR 

91.00  - 

91.00 

- 

141.00 

1.000 

19  KAZOO  PLAYER 

91.00  - 

91.00 

- 

338. 40 

1.000 

20  SINGER 

91.00  - 

91.00 

- 

180.48 

1.000 

21  DANCER 

91.00  - 

91.00 

- 

112.80 

1.000 

22  DEC 

180.00  - 

186.00 

- 

203.04 

1.000 

23 

0.00 

0.00 

NA 

0.000 

24 

0.00 

0.00 

NA 

0.000 

25 

0.00 

0.00 

NA 

0.000 

20 

0.00 

0.00 

NA 

0.000 

27 

0.00 

0.00 

NA 

0.000 

28 

0.00 

0.00 

NA 

0.000 

29 

0. 00 

0.00 

NA 

0.000 

30 

0.00 

0.00 

NA 

0.000 

31 

0.00 

0.00 

NA 

0.000 

32 

0.00 

0.00 

NA 

0.000 

33 

0.00 

0.00 

NA 

0.000 

34 

0.00 

0.00 

NA 

0.000 

35 

0.00 

0.00 

NA 

0.000 

30 

0.00 

0.00 

NA 

0.000 

37 

0.00 

0.00 

NA 

0.000 

38 

0.00 

0.00 

NA 

0.000 

39 

0.00 

0.00 

NA 

0.000 

40 

0.00 

0.00 

NA 

0.000 

ASSIGNMENTS  REMAINING  TO  COMPLETE:  2046 
TASKS  REMAINING:  1285 

ASSUMING  AVAILABLE  SKILLS  AND 

EXCLUDING  ANY  SKILLS  PRESENTLY  NOT  AVAILAOLE 

OR  NOT  APPLICABLE  ( NA >.  THE  PROJECTED 

DATE  OP  COMPLETION  IS  UEONESOAY.  SEPTEMBER  13.  1979 

THE  THREE  SKILLS  TAKING  THE  LONGEST  TIME 

WITH  THE  PRESENT  PERSONNEL  ARE.  IN  OROER:  14  22  17 


Figure  2. 
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five  major  ISD  phases:  analysis,  design, 
development,  implementation,  and  evaluation. 
Simulation  capabilities  are  specified  in 
all  five  phases;  however  they  are  most 
fundamental  to  the  programs  in  the  design 
phase.  For  example,  the  Media  Selection 
Program  assists  in  identifying  the  optimal 
delivery  medium  and  all  alternative  accep- 
table media  for  each  objective  in  the 
curriculum.  When  available  instructional 
media  are  not  clearly  defined,  this  program 
can  project  possible  alternatives.  In 
cases  where  media  changes  must  be  made,  the 
program  can  make  recommendations  based  upon 
the  instructional  requirements  of  the 
objectives.  For  example,  the  Syllabus 
Development  Program  assists  the  ISD  team  in 
sequencing  and  organizing  the  objectives, 
developing  course  maps  or  class  schedules. 

The  simulation  capabilities  involved  here 
are  the  projection  of  periods  of  peak 
resource  utilization  or  of  the  revised 
syllabi  necessitated  by  an  instructional 
resource  (e.g.,  instructor,  trainer, 
classrooms)  b“ing  unavailable. 

Thus,  AMS  and  CATSOM  illustrate  how 
simulation  can  be  helpful  at  many  different 
levels  of  the  ISD  process.  This  includes 
the  level  of  project  administration  in  terms 
of  the  effects  of  changes  in  resources  on 
budgets  and  deadlines;  the  level  of  curricu- 
lum and  instruction  where  the  varying 
effectiveness  of  different  lesson  plans, 
media  selections,  or  teaching  strategies 
can  be  explored;  and  the  level  of  ISD  modcts 
or  principles  which  involves  the  use  of 
historical  data  from  past  ISD  efforts  to 
assess  the  validity  of  proposed  ISD  analysis 
design,  development,  implementation,  or 
evaluation  models  before  they  are  applied 
to  real  projects. 

The  effectiveness  and  validity  of 
these  types  of  simulation  capabilities  will 
be  highly  dependent  upon  the  adequacy  and 
completeness  of  the  current  and  historical 
data  bases.  Thus,  the  current  data  base 
must  provide  an  accurate  picture  of  available 
resources,  status  of  the  development  of 
project  components,  specification  of  ISD 
event  sequences  and  their  resource/ personnel 
requirements  and  identification  of  the 
roles  and  skills  of  the  ISD  team.  The 
historical  data  base  represents  a cumulative 
record  of  personnel  efficiency,  resource 
utilization,  instructional  effectiveness  of 
part  syllabi  and  media  selections,  and  the 
amount  of  time  that  any  ISD  task  has 
actually  taken,  both  in  specific  instances 
and  across  all  occurrences.  The  historical 
data  base  is  clearly  a very  crucial  factor 
in  producing  valid  simulation  results;  it 
is  also  a feature  which  Is  most  often 
lacking  in  past  ISD  efforts.  Indeed,  until 


the  sort  of  management  planning  capabilities 
discussed  above  are  available,  there  has 
been  no  practical  way  to  collect  such 
detailed  historical  data  on  ISD  efforts. 

There  are  a variety  of  general-purpose 
and  even  of  training-program-specific 
simulation  systems  becoming  available.  These 
are  currently  finding  their  most  effective 
use  in  planning  and  training  support  require- 
ments analysis  of  major  programs.  Among 
these  are  such  program/systems  as  MODIA  (3), 
L-COM  (4),  and  DOSS  (5).  These  programs, 
while  they  reflect  varying  degrees  of 
sophistication  and  power,  do  not  offer  the 
solutions  and  support  required  for  the 
real-time  or  on-line  management  of  major  ISD 
activities.  Characteristics  of  the  manage- 
ment systems  which  are  required  now,  and 
which  will  be  required  in  the  near  future, 
are  fairly  straight  forward.  Due  to  the 
compressed  time  frames  of  most  ISD 
activities  and  the  great  number  of  short- 
term deadlines  and  simultaneous  activities 
being  undertaken,  the  system  must  be  real- 
time and  should  support  the  capability  for 
routine  data  collection,  project  monitoring, 
and  report  generation  activities  at  the 
same  time  that  they  support  the  types  of 
management  simulation  activities  discussed 
here.  Simulation  capabilities  being  added 
to  the  new  generation  of  training  systems 
such  as  the  Navy  VTS  (6)  or  the  Air  Force 
AIS  (7)  will  probably  meet  these  require- 
ments. 

These  ISD  simulation  systems  must  have 
a number  of  essential  features.  One 
important  characteristic  of  these  systems 
is  that  they  should  be  human  engineered 
for  users  such  as  project  managers, 
secretaries,  psychologists  and  ISD  team 
personnel,  not  computer  programmers.  A 
counter-example  here  might  be  MODIA,  one 
of  the  most  powerful  and  flexible  of  the 
simulation/projection  systems  currently 
available.  The  use  of  MODIA  requires  at 
least  two  expert  teams  of  personnel.  One 
team  of  personnel  should  be  experts  in  the 
system  being  analysed  and  the  other  team 
must  be  specifically  trained  and  experi- 
enced in  the  use  of  the  MODIA  program  itself. 
In  other  words,  this  system,  while  powerful 
and  sophisticated  is  not  human  engineered 
to  the  class  of  users  involved  in  most  ISD 
activities. 

It  is  also  important  that  the  simula- 
tion and  planning  capabilities  be  integrated 
into  a comprehensive  ISD  management  system 
such  as  the  CATSDM  project.  Attempting  to 
add  simulation  or  modeling  capabilities  in 
a piecemeal  fashion  can  result  in  a Tower 
of  Babel  from  a programming  and  data  base 
organization  standpoint  and  typically  makes 
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the  system  cumbersome  and  inefficient.  This  (4)  L-COM:  RM-5544-PR,  May  1968 

argues  for  the  general  point  that  simulation  The  Rand  Corporation 

must  eventually  be  accepted  as  an  integral 

component  of  the  ISD  management  process  (5)  DOSS:  R-1435-PR,  July  1974 

DOSS:  R-1610-PR,  January  1975 
The  Rand  Corporation 
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MYOELECTRIC  FEEDBACK  CONTROL  OF  ACCELERATION  INDUCED 
VISUAL  SCENE  DIMMING  IN  AIRCRAFT  TRAINING  SIMULATORS 


D.  ROGERS  and  D.  QUAM 
University  of  Dayton 

This  study  is  being  conducted  under  contract  with  the 
Air  Force  Human  Resources  Laboratory 


SUMMARY 

Visual  dimming  experienced  during  air- 
craft maneuvering  accelerations  is  an  im- 
portant information  source  for  the  simulator 
pilot.  A method  for  integration  of  the  man 
with  the  simulator  is  demonstrated  using 
computer-based  physiologic  models  and 
readily  measured  electromyographic  signals 
arising  from  the  straining  maneuver.  This 
technique  forces  active  participation  and 
energy  expenditure  by  the  simulator  pilot 
similar  to  the  requirements  of  the  aircraft 
pilot  undergoing  acceleration. 

Simulation  models  are  developed  which 
relate  the  effects  of  the  G suit,  straining 
techniques  and  the  cardiovacsular  response 
system  to  a single  protection  variable  (PV). 
The  PV  signal  drives  a predictive  visual 
field  model  derived  from  analysis  of  the 
retinal  circulation  in  the  eye.  Allowances 
are  made  for  subject  variation  and  cockpit 
seating  configurations. 

The  model  produces  accurate  predic- 
tions of  short-term  Gz  tolerance  in  its  pre- 
sent form  and  with  slight  modifications  can 
be  adapted  to  include  energetic  costs  for 
long-term  accelerations. 

INTRODUCTION 

The  aircraft  pilot  in  causing  his  plane  to 
change  its  velocity  vector  places  himself  in 
a changing  acceleration  environment.  At 
certain  levels,  durations  and  directions  of 
acceleration  the  visual  field  of  the  pilot  is 
diminished,  possibly  to  the  point  of  black- 
out. At  greater  levels  and  durations  uncon- 
sciousness may  occur.  The  greatest  level 
at  which  the  pilot  still  has  vision  is  referred 
to  as  his  acceleration  tolerance.  In  flight, 
he  has  both  protective  measures  and  devices 
which  improve  this  acceleration  tolerance. 
Straining  and  grunting  during  tight  turns 


were  practiced  by  German  pilots  prior  to 
World  War  II  as  a means  of  improving  toler- 
ance. This  practice  led  to  the  development 
of  the  M- 1 maneuver.  Other  techniques 
using  posture  changes,  restraint  systems, 
and  body  position  have  been  well  explored. 
Present  G protective  techniques  (principally 
the  G suit)  attempt  to  prevent  the  conse- 
quences of  the  peripheral  pooling  of  the 
blood  during  acceleration  by  the  application 
of  a suitable  counter-pressure. 

The  simulator  pilot  does  not  have  to  con- 
tend with  the  actual  acceleration  forces  in 
his  ground  maneuvers.  The  G suit  and  the 
G seat  provide  a tactile  feel  of  the  accelera- 
tion, however  the  internal  visual  field  of  the 
simulator  pilot  is  not  impaired  nor  does  he 
need  to  strain.  The  muscular  straining  re- 
quirements can  be  included  in  the  simulation 
scenario  so  that  the  pilot  experiences  a 
more  realistic  training  situation  in  the  simu- 
lator. Thus,  it  is  possible  to  create  an 
accurate  representation  of  visual  field 
dimming  and  fatigue  onset  for  the  simulator 
pilot.  The  G effects  can  be  simulated  real- 
istically by  integrating  the  man  with  the 
simulator  system.  This  integration  can  be 
made  by  a combination  computer-based  dynamic 
physiologic  models  and  easily  instrumented 
myoelectric  feedback. 


PHYSIOLOGIC  FACTORS  OF 
ACCELERATION  TOLERANCE 

The  physiologic  changes  caused  by 
acceleration  have  been  reported  on  exten- 
sively in  research  literature.  The  two 
widely  recognized  factors  which  contribute 
to  pilot  impairment  in  maneuvering  aircraft 
are  fatigue  and  visual  dimming.  These  two 
factors  can  be  included  in  a simulation 
model  system  to  enhance  the  reality  of 
simulated  air  combat. 
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The  cardiovascular  system  has  received 
the  greatest  attention  in  acceleration  stress 
studies  as  it  provides  an  easily  measured 
set  of  symptoms.  The  limiting  factor  in  the 
cardiovascular  system  from  the  viewpoint  of 
combat  maneuvering  stress  is  the  reduction 
of  blood  pressure  at  eye  level  and  subsequent 
loss  of  vision.  The  pressure  gradient 
changes  caused  by  the  G loading  and  blood 
pooling  are,  of  course,  the  major  contributing 
factors  which  reduce  the  pressure  available 
at  the  eye  and  in  more  severe  cases  at  the 
brain.  Counteracting  the  reduced  pressure 
is  the  primary  purpose  of  straining  maneu- 
vers, the  M- 1 and  L-l.  Blood  pooling  in  the 
lower  body  is  reduced  by  use  of  the  G suit. 

The  factors  then,  which  define  an  operating 
limit  in  the  cardiovascular  system,  are  re- 
lated directly  to  the  blood  pressure  response 
characteristics  in  the  3-15  second  time 
frame. 

Another  area  of  interest  in  the  physiolog- 
ic response  to  G stress  is  the  time  period 
beyond  15  seconds.  Once  the  pilot  has 
passed  a G stress  level  which  represents  his 
relaxed  tolerance,  he  must  expend  energy  in 
straining  to  increase  blood  pressure  and,  thus, 
maintain  vision.  The  amount  of  energy  ex- 
pended in  the  blood  pressure  maintenance 
task  is  a factor  of  his  straining  efficiency 
and  the  magnitude  of  the  G difference  be- 
tween his  relaxed  tolerance  and  the  current 
G stress  level.  It  can  be  speculated  that  the 
time  endurance  limit  is  a factor  of  the  indi- 
viduals available  energy  pool  minus  the 
energy  used  in  visual  maintenance.  When 
the  energy  pool  is  diminished  to  a certain 
level,  the  fatigue  limit  is  reached.  One  pur- 
pose of  the  pro,  osed  simulation  system  is  to 
cause  the  simulator  pilot  to  be  energetically 
loaded  in  the  same  manner  as  the  aircraft 
pilot. 

SYSTEM  STRUCTURE 

The  system  presented  on  the  following 
pages  represents  the  results  of  nartitioning 
the  complex  physiologic  system  into  linear 
sub  models.  Each  sub  model  is  explored  and 
developed  in  detail  based  on  a common  pro- 
tection variable  PV  which  is  related  physio- 
logically to  system  blood  pressure.  The 
blood  pressure  models  are  then  finally  com- 
bined in  a systematic  paradigm  which  pro- 
vides the  driving  values  for  the  visual  field 
response  model.  There  are  four  seperable 


models  which  result  from  the  partitioning 
process;  the  cardiovascular  model,  the 
straining  model,  the  G suit  model,  and  the 
visual  field  model. 

The  cardiovascular  response  model  is 
represented  by  a dynamic  linear  transfer 
function  which  corresponds  with  the  response 
of  the  pilots  blood  pressure  to  acceleration. 
The  governing  factor  in  pilot  response  to 
acceleration  is  the  onset  of  greyout  and 
blackout.  These  visual  problems  are  direct- 
ly related  to  the  available  blood  pressure  at 
eye  level.  This  model  output  provides  a 
dynamically  responding  signal  which  is  equi- 
valent to  nominal  eye  level  blood  pressure 
values  for  a human  undergoing  the  equivalent 
acceleration,  G(t),  profile. 

The  straining  simulation  model  accounts 
for  the  G tolerance  enhancement  which  is 
afforded  by  a properly  executed  M-  1 maneu- 
ver. The  purpose  of  the  straining  or  M- 1 
maneuver  is  to  increase  the  blood  pressure 
delivered  to  the  eye.  Proper  performance 
of  the  maneuver  requires  that  the  abdominal 
and  upper  torso  muscles  be  tensed  isomet- 
rically  and  that  expirations  should  be  made 
against  a closed  glottis.  The  result  is  an 
increased  intrathoracic  pressure  and  in- 
creased blood  pressure  at  the  eye.  Proper 
application  of  the  straining  maneuver  results 
in  the  appearance  of  myoelectric  signals  on 
the  skin  surface.  These  biologically  derived 
signals  are  processed  by  the  model  to  gener- 
ate a straining  protection  variable  PVs 
which  represents  the  increased  blood  pres- 
sure due  to  the  M-  1. 

The  pressurized  G suit  is  an  important 
protective  garment  used  to  increase  the  in- 
dividuals tolerance  to  +GZ.  The  suit  uses 
pressurized  bladders  to  press  against  the 
legs  and  lower  abdomen.  The  external  pres- 
sure inhibits  displacement  of  the  blood 
volume  to  the  lower  extremities  thus  insur- 
ing a better  supply  to  the  heart  during  accel- 
eration. The  suit  must  be  inflated  by  the  G 
valve  to  a predetermined  level  to  be  effec- 
tive. The  simulation  model  accounts  for  the 
required  pressure  level  and  uses  the  actual 
suit  to  provide  the  necessary  dynamics.  The 
suit  pressure  is  compared  with  the  required 
schedule  and  a protection  value  is  generated 
by  the  model. 

The  dynamic  visual  field  model  is  devel- 
oped to  be  readily  implemented  in  a 


simulation  system.  The  visual  field  model 
reacts  to  PV  level  inputs  from  an  external 
source  and  produces  a dynamically  respon- 
sive signal  which  predicts  the  expected 
visual  field  of  a pilot  undergoing  the  identical 
G profile. 

The  required  dynamic  model  of  pilot 
visual  response  to  Gz  assembled  as  a super- 
position of  the  separate  models.  The  com- 
pleted system  algorithms  provide  the  means 
for  implementation  in  aircraft  training  sim- 
ulators. The  integrated  system  is  structured 
according  to  Figure  1. 

Pilot  commands  thru  the  simulator 
aircraft  dynamics  cause  changes  in  the  air- 
craft velocity  vector.  Vector  changes  which 
result  in  a linear  component  of  +Gz(t)  are 
used  to  drive  the  cardiovascular  model  ©• 
In  addition  the  Gz(t)  signal  causes  the  pilots 
G suit  (4^  to  inflate  according  to  a 


preselected  schedule.  The  effect  of  the 
proper  suit  inflation  is  to  increase  the  value 
of  the  protective  variable  within  certain 
limits.  The  sum  of  these  signals  ($)  is 
added  with  a random  signal  related  to  indi- 
vidual variation.  The  resulting  PV  signal 
drives  the  visual  field  model.  The  instru- 
ment panel  and  the  view  screen  are  then 
dimmed  according  to  the  output  of  the  model 
(6^.  The  pilot  senses  the  size  and  bright- 
ness changes  and  has  the  option  of  reducing 
his  aircraft  maneuver  intensity  or  increas- 
ing his  G tolerance  level  by  performing  an 
M- 1 maneuver.  If  he  initiates  the  M- 1 
maneuver,  the  straining  model  processes 
the  myoelectric  activity  from  his  skin  sur- 
face and  modifies  it  wift  a signal  related  to 
the  straining  interval  QO  • If  he  performs 
the  appropriate  action  , the  mo£el  responds 
with  an  additive  value  of  PV  (i)  and  his 
visual  scene  is  enhanced.  The  separate  mo- 
dels are  described  in  the  following  sections. 


Figure  1.  System  Diagram 


i 
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THE  CARDIOVASCULAR  MODEL 


Human  tolerance  to  long-term  +GZ  ac- 
celeration is  normally  measured  in  terms 
of  visual  loss  (blackout)  and  unconscious- 
ness. Both  of  these  tolerance  end  points  are 
related  to  the  ability  of  the  cardiovascular 
system  to  deliver  oxygenated  blood  at  ade- 
quate pressure  to  the  retinal  and  cerebral 
regions.  The  distribution  of  the  blood  in  the 
body  also  changes  as  the  acceleration  pools 
blood  in  the  lower  parts  of  the  body  and 
lungs.  There  is,  therefore,  less  available 
blood  to  circulate  and  a lower  oxygen  con- 
tent because  the  lungs  do  not  operate  as 
efficiently. 

There  are  two  cardiovascular  systems 
which  are  dynamically  involved  in  the  pro- 
cess of  blood  pressure  maintenance  while 
the  human  is  undergoing  +GZ  acceleration. 
The  hydrostatic  system  which  is  related  to 
classical  fluid  mechanics  is  responsible  for 
the  reduced  retinal  perfusion  pressure  at 
the  eye  and  eventual  loss  of  pressure  at  the 
cerebral  level.  The  orthostatic  system  is 
related  to  blood  pooling  in  the  lower  body 
with  a concomitant  reduction  to  venous  re- 
turn to  the  heart.  The  cardiovascular  sys- 
tem has  self-regulatory  feedback  systems 
which  are  affected  by  blood  volume  and  pres- 
sure. The  feedback  mechanisms  attempt  to 
regulate  the  pressure  and  flow  characteris- 
tics of  the  cardiovascular  system. 

Both  animal  analogs  (principally  canine) 
and  human  experimentation  have  led  to  the 
current  knowledge  about  the  blood  pressure 
response  to  +GZ.  Canine  blood  pressure 
response  curves  have  been  derived  by  Knapp 
with  maximum  gain  in  the  30  to  60  mHz 
range.  Koushanpour  et  al.  show  a first 
order  transfer  function  with  a 20  second 
time  constant.  Levison  has  proposed  a 
second  order  system  for  the  canine  carotid 
reflex  with  a natural  frequency  of  42  mHz. 
Examination  of  Gillinghams  data  as  a Bode 
plot  indicates  that  a single  zero  double  pole 
transfer  function  can  be  applied  with  reason- 
able results.  This  study  has  adopted  a like 
function  to  represent  the  Pressure- Gz  accel- 
eration (P-G)  transfer  function.  The  fre- 
quency range  of  interest  is  restricted  to 
f 200  mHz  as  the  physiologic  responses  of 
interest  to  long-term  maneuvering  accelera- 
tions falls  within  this  range.  The  general- 
ized transfer  function  is: 


K (1+a  s) 

P(s)  j 

1 + bjS  + b2s  ' 

Values  are  selected  for  aj,  bj  and  63  as 
compromise  values  from  the  literature  and 
Gillinghams  response  curves.  The  system 
lead  term  is  selected  at  fz  = 30  mHz  and  the 
system  response  is  selected  as  a complex 
pole  at  fp  = 70  mHz  and  5 = 0.  7.  For  these 
breakpoints  the  values  of  aj  = b.  31,  b]  = 

3.23  and  b£  = 5.  17. 

The  blood  pressure  transfer  function 
gain  at  eye  level  is  directly  affected  by  the 
seat  back  angle,  the  direction  of  the  local 
vertical  G axis  and  the  anatomic  offset  of  the 
eye.  The  static  pressure  at  eye  level  is  cal- 
culated from 

Pae  = Pa  - . 77  he  cos  9 G 

where  Pae  and  Pa  represent  pressure  at  the 
eye  and  heart  level  respectively,  and  the  ef- 
fect of  the  seat  angle  offset  is  implemented 
by  a modification  of  the  cardiovascular  trans- 
fer function  gain  K j ( © ).  The  transfer  func- 
tion gain  is  then  given  as 


Pae  - Pa 
G 


K j (0  ) = -.  77  he  cos  9. 


The  offset  angles  are  shown  in  Figure  2 
where  the  seat  back  is  defined  as  coincident 
with  the  physiologic  z axis  Pz. 


For  normal  operation  the  transfer  func- 
tion from  G to  blood  pressure  takes  the  form 


P(s)  = -21.4  cos  9 


1 + 5.  31s 

1 + 3.  23s  + 5.  17sZ 


THE  G SUIT  MODEL 

The  history  of  G suits  dates  from  World 
War  II  and  is  well  reviewed  in  recent  mono- 
graphs. The  wraparound  CSU-3/P  cutaway- 
type’ of  anti-G  suit,  presently  used  by  the 
U.  S.  Air  Force,  improves  blackout  toler- 
ance in  the  +GZ  vector  by  about  2 G above 
resting  tolerance. 

The  relationship  between  the  G suit 
pressurization  and  effective  blood  pressure 


I 

IL 
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Figure  2.  Physiological  Axes  with 
Anatomical  Offsets 


6s  - seat  angle  6e  - retinal  angle 

Gz  - local  gravity  Pz  - Physiologic  z axis 
he  - eye  height  above  aorta  6 - G axis  offset 

increase  at  eye  level  is  highly  complex. 

The  physiological  mechanism  of  action  of 
anti-G  suits  was  originally  established  on 
the  Toronto  centrifuge  and  at  the  Mayo 
Clinic  and  has  been  summarized  by  Wood 
and  Lambert.  They  show  that  inflation  of  a 
G suit  at  1G  produces  an  initial  increase  in 
arterial  pressure,  followed  by  an  almost 
immediate  decrease  in  heart  rate  probably 
due  to  a depressor  reflex  originating  in  the 
carotid  sinus  and  aortic  areas. 

McCally  has  shown  that  the  G suit  does 
not  provide  protection  unless  the  suit  is 
inflated  to  at  least  80  mmHg.  In  effect,  the 
pressure  difference  between  the  suit  bladder 
and  the  hydrostatic  blood  pressure  in  lower 
body  must  be  such  that  the  suit  pressure  is 
greater  to  effect  a protection  function.  The 
same  reasoning  is  used  in  developing  the 
simulation  model.  The  suit  pressure  must 
be  within  acceptable  pressure  tolerance  or 
it  will  afford  no  protection  value. 


The  protective  G suit  garment  contains 
air  tight  bladders  which  are  filled  with  pres- 
surized air  delivered  from  a G sensitive 
mechanical  valve.  The  pressure  delivered 
by  the  valve  is  a function  of  the  current  G 
level. 

G suit  inflation  does  begin  when  the 
valve  has  reached  a level  of  1.  5 - 1.7  G. 
After  this  point  the  pressure  output  is  a 
linear  function  of  G with  a value  of  approxi- 
mately 75  mmHg  per  G.  The  air  bladders, 
the  suit  air  feed  hose  and  the  containing  gar- 
ment represent  a dead  space  which  intro- 
duces a time  delay  into  the  pressure  system. 

The  blood  pressure  G suit  model 
assumes  a nominal  2G  or  equivalent  42.  8 
mmHg  increase  for  a properly  inflated  suit. 
For  a suit  with  lower  pressure  than  designat- 
ed the  increased  G protection  and  equivalent 
blood  pressure  value  decay  linearly  to  0. 

The  model  is  driven  by  a delta  P (AP)  repre- 
senting the  pressure  difference  between  the 
standard  suit  pressure  curve  and  the  actual 
suit  pressure.  When  suit  pressure  is  equal 
to  or  greater  than  the  standard  curve  value, 
full  protection  is  assumed.  When  the  suit 
pressure  is  less  than  required  by  the  stan- 
dard curve,  the  protection  value  is  lowered. 
When  the  pressure  differential  is  greater 
than  80  mmHg  with  the  suit  pressure  below 
required,  there  is  no  protection  afforded  by 
the  simulation  model. 

Where  AP  = Suit  Pressure  - Standard 

Pressure 


For  AP  s - 80  Protection  Value  0 
For  -80  < AP<  0 PV  = P = 42.  8 

oU 

For  0 s AP  PV  = 42.  8 

The  G suit  dynamic  response  characteris- 
tics are  included  in  the  simulation  automat- 
ically as  the  G suit  pressurization  for  the 
simulator  pilot  provides  the  driving  signal 
to  determine  the  protection  value.  Figure 
3 is  a block  diagram  representation  of  the 
G suit  model. 


r 


Figure  3.  0 Suit  Model 


THE  STRAINING  MODEL 

Since  the  early  days  of  flying,  the  value 
of  muscular  straining  as  protection  against 
the  effects  of  +GZ  acceleration  have  been  re- 
cognized. Pilots  observed  that  blackout  or 
greyout  could  be  postponed  if  they  grunted  or 
screamed  and  tensed  the  skeletal  musculature 
during  acceleration.  The  M-  1 maneuver,  de- 
fined as  muscular  straining  with  expiration 
against  a partially  closed  glottis,  is  a most 
effective  voluntary  protection  against  the  cir- 
culatory effects  of  +GZ  acceleration.  The 
physiologic  basis  for  this  protection  has  been 
ascribed  to  its  effect  on  increasing  the  arteri- 
al pressure  at  eye  during  acceleration.  As 
forced  expiration  is  instituted,  the  resulting 
increase  in  intrathoracic  pressure  is  trans- 
mitted directly  to  the  aorta  and  a like  in- 
crease is  felt  at  eye  level. 

Skeletal  muscle  is  controlled  by  signals 
which  are  transmitted  to  selected  motor  units 
of  a muscle  through  the  motor  neurons.  The 
force  generated  by  the  muscle  is  the  result 
of  both  the  frequency  of  firing  of  motor  units 
and  the  number  of  motor  units  which  are  re- 
cruited. The  muscle  tension  is  accompanied 
by  electrical  signals  which  can  be  detected  by 
suitable  electrodes  on  the  skin  surface.  The 
electrical  signal  exhibits  the  characteristics 
of  its  primary  source  in  that  it  represents  a 
weighted  sum  of  motor  unit  activations. 

There  is,  therefore,  a correspondence  between 
the  EMC  and  muscle  force. 

The  muscle  straining  maneuvers  such  as 
the  M-  1 and  L-  1 require  a general  tensing  of 


skeletal  muscle  along  with  contraction  of 
abdominal  and  peripheral  muscles.  The  mus- 
cle straining  is  accompanied  by  the  expira- 
tion of  air  through  a partially  closed  glottis 
for  the  M-l.  Both  can  cause  increased  in- 
trathoracic pressure  of  50  to  100  mmHg  and 
consequently  increase  arterial  blood  pres- 
sure by  a like  amount. 

This  simulator  model  requires  the  pre- 
sence of  two  signals  to  provide  the  maximum 
straining  protection  value,  PVs.  The  sig- 
nals represent  the  prime  factors  which  are 
present  in  a properly  executed  straining 
maneuver.  The  EMG  signal  is  generated  by 
the  straining  subject  and  processed  to  pro- 
vide and  intermediate  protection  value  PVm. 
The  protection  afforded  by  muscle  straining 
is  then  modified  by  the  timing  of  the  strain- 
ing maneuver. 

The  thoracic  cavity  is  modelled  as  a 
flexible  cylinder  surrounded  by  a muscle 
girdle.  The  muscle  straining  signal  repre- 
senting muscle  force  around  the  thoracic 
volume  is,  therefore,  assumed  to  bear  a pre- 
dictable relation  to  the  internal  pressure 
increase  at  low  straining  levels.  At  higher 
levels  a maximum  pressure  is  achieved  and 
increases  in  muscles  straining  are  no  longer 
effective.  The  pressure  straining  relation- 
ship used  in  the  model  is  a curve  with  a 
saturation  level  related  to  the  maximum 
pressure  rise  in  the  thoracic  cavity. 

The  effectiveness  of  the  straining  maneu- 
ver is  also  governed  by  the  repetition  rate. 

The  straining  protection  value  K2<tr)  is, 
therefore,  modified  by  a function  dependent 
upon  maneuver  repetition  rate.  Experimen- 
tally, K2<tr)  is  maximum  at  3-5  seconds 
according  to  Gillingham.  Shorter  time 
periods  do  not  allow  sufficient  time  for  the 
internal  pressure  to  rise  and  longer  time 
periods  involve  countering  responses  due  to 
baroreceptor  feedback  and  reduced  venous 
return  to  the  heart.  The  generalized  strain- 
ing simulation  block  diagram  is  shown  in 
Figure  4. 

VISUAL  LIMITS  MODEL 

The  effect  that  acceleration  has  on  the 
visual  apparatus  is  observed  in  terms  of 
tunnel  vision,  greyout,  and  blackout.  During 
the  periods  when  vision  is  impaired  there 
are  also  decreases  in  visual  activity;  and 
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Figure  4.  Straining  Model 


brightness  contract  detection  ability.  Al- 
though there  are  multiple  factors  related  to 
the  anatomy,  psychology  and  physiology  of 
the  human  which  are  responsible  for  these 
changes  in  visual  perception,  the  structure 
of  the  eye  is  a primary  factor  and  provides 
the  basis  for  a useable  model. 

Consider  the  monocular  field  with  the 
visual  center  located  at  the  fovea,  and  the 
arterial  supply  offset  and  entering  the  retinal 
surface  through  the  optic  disk.  The  arterio 
lar  branches  which  perfuse  the  retina  spread 
out  over  the  retina  terminating  in  the  capil- 
laries, Figure  5.  The  major  arterial 


Figure  5.  Retinal  Artery  Supply  on  Right 
Eye  Field 


branches  course  across  the  retina  to  form  a 
general  network  which  normally  provides  an 
adequate  O2  blood  supply  to  the  retina.  The 
arterial  supply  is  formed  so  that  a nominal 
pressure  drop  is  encountered  as  the  distance 
from  the  optic  nerve  entrance  is  increased. 
Figure  6. 


F-. jure  6.  Linearized  Representation 
of  Blood  to  Retina 


When  the  pressure  drops  below  a critical 
value,  the  more  distal  areas  begin  to  feel  the 
effects  of  blood  and  oxygen  depletion.  When 
the  pressure  at  the  retinal  artery  drops  be- 
low the  interocular  pressure  blood  flow 
ceases  and  the  retina  is  no  longer  capable  of 
transmitting  neural  signals  due  to  light  detec- 
tion in  the  rods  and  cones. 

As  an  initial  approximation  the  retinal 
supply  network  is  assumed  to  be  a linear  net- 
work with  the  supply  pressure  distributed 
linearly  across  the  network.  That  is  at  any 
point  along  the  supply 


Where  dx  is  the  distance  along  the  arteriolar 
bed  dP  is  the  corresponding  pressure  change 
along  dx  and  Kj  is  a function  of  the  central 
supply.  At  some  critical  pressure  blood 
flow  ceases  with  the  peripheral  areas  experi- 
encing the  initial  blood  flow  shut  off.  The 
supply  of  blood  and  oxygen  then  decays  in- 
wardly toward  the  supply  point  and  the  visual 
sensitivity  of  the  system  decays  in  a like 
manner. 

The  visual  field  model  can  now  be  ex- 
tended to  a binocular  field.  Figure  7 is  a 
binocular  field  map  showing  the  coincident 
foveal  areas  as  the  visual  center,  and  the 
optic  disc  for  each  eye  in  a manner  that  de- 
picts the  observed  field  from  inside  out. 

The  outer  lines  depict  the  outer  edges  of  the 
peripheral  vision.  Lowered  blood  pressure 
supply  in  each  of  the  modeled  retinas  causes 
the  field  to  collapse  as  concentric  circles 


with  centers  at  each  of  the  optic  discs  as 
shown  by  the  dashed  lines.  Thus  the  visual 
field  collapses  toward  a somewhat  ellipsoi- 
dal shape  with  the  vertical  field  having  a 
smaller  visual  angle  than  the  horizontal  field. 


Figure  J.  Binocular  Visual  Field  Decay  Due  to 
Reduced  Blood  Supply  Pressure 


CONCLUSIONS  AND  RECOMMENDATIONS 

The  models  presented  in  this  study  are 
designed  for  implementation  in  real-time 
simulation.  As  a consequence  many  simpli- 
fying assumptions  have  been  made.  The 
models  do  not  represent  the  final  stage  of 
myoelectric  integration  but  do  provide  a logi- 
cal structure  from  which  refinement  can  ad- 
vance. There  are  gaps  in  the  information 
available  which  must  be  filled  to  further  re- 
fine the  models  and  increase  simulation 
fidelity. 

Reevaluation  of  the  visual  model  indicates 
that  a dynamic  model  of  the  blood  distribution 
system  across  the  retina  may  be  very  valu- 
able. A model  analog  can  be  seen  in  transmis- 
sion line  theory  and  could  provide  a single 
model  for  both  peripheral  and  central  visual 
fields. 


A promising  extension  of  the  visual 
model  would  include  a dynamic  representa- 
tion of  oxygen  transport  at  the  synapse 
junctions  in  accordance  with  Miller  and 
Green.  Reduction  of  oxygen  concentration 
levels  due  to  impaired  pulmonary  function 
or  exercise  would  play  an  important  role  in 
this  extended  model.  The  expected  outcome 
would  yield  zonal  gradients  of  contrast  and 
acuity  limits  as  a function  of  circulatory 
distance  from  the  retinal  supply.  In  the  ex- 
tended model  both  the  available  oxygen  in 
the  blood  and  the  blood  pressure  would  be 
important  factors  in  defining  the  visual 
field  response. 

The  extended  model  could  provide  defi- 
nition of  simulator  display  requirements  by 
establishing  maximum  acuity  and  minimum 
brightness  requirements  for  optimal  scene 
projection  fidelity  based  on  visual  capability. 
The  extended  model  would  also  allow  accur- 
ate manipulation  of  the  generated  scene  in 
terms  of  both  acuity,  contrast  and  relative 
brightness. 

Considerable  benefit  would  result  from 
an  experimental  procedure  to  determine  the 
dynamic  relationship  between  EMG  and  ele- 
vated blood  pressure. 

1 ) Experimental  verification  of  the  derived 
transfer  function  would  provide  the  basis 
for  refinement  of  the  postulated  model. 

2)  An  evaluation  of  the  energetic  costs  of 
the  straining  protective  maneuvers  could  be 
derived  from  such  a study  and  when  incor- 
porated into  the  current  simulation  model 
would  provide  a new  method  for  evaluation 
of  acceleration  protection  equipment. 

Further  exploration  of  electrode  place- 
ment should  include  the  possibility  of  using 
seat  pan  electrodes  (recommended  by  K. 
Gillingham).  This  method  would  obviate  the 
need  for  separate  electrode  preparation  for 
each  subject  and  provide  the  maximum  in 
environmental  fidelity.  The  simplifying 
assumptions  used  in  linearizing  the  complex 
physiologic  systems  are  both  scientifically 
and  pragmatically  motivated.  The  reduction 
of  assumption  to  fact  for  these  models  will 
require  a wide  range  of  experimental  efforts. 
However,  the  system  points  in  the  new  direc- 
tions, provides  the  rationale  and  establishes 
the  feasibility  of  biologic  integration  in 
training  systems. 
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MODIFICATION  OF  DEFENSE  MAPPING  AGENCY 
AEROSPACE  CENTER  (DMAAC)  DIGITAL  RADAR 
LANDMASS  SIMULATOR  (DRLMS)  DATA  BASE 


FORTUNATA  V.  ALTMAYER 
GRUMMAN  AEROSPACE  CORPORATION 
BETHPAGE,  NEW  YORK  11714 


SUMMARY 


A planar  approximation  for  data  compres- 
sion of  DRLMS  data  base  is  provided  by  Gen- 
eral Electric.  The  data  considered  were 
the  terrain  elevation  slope  between  adjacent 
points  within  a 1°  X 1°  rectangle.  Because 
of  the  heterogeneity  such  a large  area, 
smaller  areas  1/1600  of  the  1°  X 1°  rectangle 
were  considered  and  the  planar  approximation 
applied  to  each  of  the  smaller  rectangles. 

To  substantiate  the  need  for  a reduction  of 
the  1°  X 1°  area,  a roughness  index  for  each 
of  the  1600  blocks  in  the  area  in  question 
was  calculated. 

The  Grumman  modification  of  the  General 
Electric  "Trig"  (Triangle  Generation)  algo- 
rithm which  implements  this  planar  approxima- 
tion reduces  computational  time  and  enhances 
the  fidelity  of  the  simulated  video  signal. 

DISCUSSION 

The  Digital  Radar  Landmass  Simulator 
(DRLMS)  simulates  the  ground  mapping  of  all 
modes  of  the  AN/APQ-156  Radar.  It  is 
required  that  this  simulator  be  realistic 
enough  to  the  ground  training  environment  to 
permit  substitution  of  simulator  time  for 
flight  training  time.  A combined  effort  of 
General  Electric  and  Defense  Mapping  Agency 
Aerospace  Center  (DMAAC)  produced  a system, 
currently  incorporated  in  A6E-WST,  in  which 
DRLMS  computes  radar  video  signals  for  each 
resolvable  element  of  the  earth's  surface 
illuminated  by  the  radar  antenna.  The  DMAAC 
off-line  digital  data  base  contains  terrain 
relief  data  that  was  digitized  as  the  eleva- 
tion values  at  the  intersections  of  a grid 
and  also  the  cultural  descriptive  data  that 
likewise  was  digitized  onto  a grid.  Stored 
terrain  and  cultural  data  from  the  DMAAC  data 
base  is  used  for  computing  the  radar  receiver 
output  and  this  information  is  used  to  drive 
the  radar  display.  It  is  specifically  the 
terrain  file  to  which  this  report  addresses 
itself.  The  DRLMS  employs  three  range  depen- 
dent on-line  data  bases  to  simulate  radar 
images;  these  are: 


o Long  Range  (75  n.  mi.  - 150  n.  mi.) 

o Medium  Range  (27.5  n.  mi.  - 75  n.  mi.) 

o Short  Range  (5  n.  mi.  - 27.5  n.  mi.) 

To  generate  the  data  base  for  the  DRLMS, 
the  grid  terrain  model  is  defined  by  DMAAC 
and  a planar  approximation,  used  for  data 
compression,  is  provided  by  GE.  The  planar 
technique  for  terrain  modeling  is  a variable 
compression  method — planar  segments  being 
very  large  in  areas  where  the  world  is  flat 
and  such  segments  being  small  where  the  ter- 
rain is  rugged.  With  this  variation,  such  an 
approach  should  allow  placement  of  the  data 
where  it  is  needed. 

A.  General  Electric  Basic  Trip  Algorithm 

The  planar  approximation,  implemented  in 
G.E.'s  "Trig"  (Triangle  Generation)  Algo- 
rithm, adjusts  range  dependent  parameters 
not  only  to  satisfy  the  overall  terrain  ele- 
vation but  also  to  be  self-adaptive  to  the 
memory  system  capacity.  A major  disadvantage 
of  this  technique  is  that  the  parameters  are 
only  range  dependent  and  not  terrain  quality 
dependent.  It  is  to  rectify  this  insuffi- 
ciency that  the  Grumman  modification  was 
initiated. 

A typicalfiGE  selected  data  base  repre- 
sented 1.6  X 10b  square  miles,  or  1600  31 
element  X 31  element  arrays.  The  actual  ter- 
rain elevation  for  each  array  position 
(DMAAC)  is  compared  with  the  elevation  on  the 
plane  (GE  derived)  for  the  same  X-Y  location, 
and  the  absolute  value  of  this  difference  is 
calculated.  Then,  the  mean  deviation  of 
these  differences  is  computed.  If  the 
absolute  difference  of  any  X-Y  loca- 
tion exceeds  some  input  parameter,^,  or  if 
the  mean  deviation  of  the  planes  exceeds  the 
input  parameter, £ , the  model  is  rejected, 
and  a new  model  is  constructed.  While  the 
(•^  , £ ) parameters  were  selected  for  31x31 
blocks,  1600  such  blocks  representing  a one 
by  one  degree  area,  were  compiled  and 
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investigated  in  their  entirety.  Only  one 
parameter  set  applied  to  the  block  conglom- 
erate. Such  a philosophy  is  adequate  if  the 
terrain  of  these  1600  blocks  is  homogeneous. 
For  areas  of  heterogeneity,  however,  the  data 
compression  technique  produces  a compromised 
data  base  that  does  not  meet  the  subjective 
requirements  for  smooth  areas  within  the 
heterogeneous  area. 

If  the  original  31x31  word  array  is  re- 
jected, a new  model  is  formed  by  dividing 
the  31x31  array  into  four  16x16  subarrays 
with  two  planes  for  each. 

31  elements 

t | 1 

— ( 

31  elements 

I 


Figure  1:  16x16  subarrays 


Again,  each  of  the  16x16  element  subarrays,  is 
investigated  to  determine  its  passing  the  , 
£ criterion.  If  failure  exists,  each  sub- 
array is  further  subdivided  and  again  tested. 
If  less  than  three  of  the  quadrants  fails, 
the  array  is  subdivided  into  nine  mini-arrays 
(Fig.  2)  and  then  twenty-five  mini-arrays 
(Fig.  3).  If,  then,  any  of  the  twenty-five 
mini-arrays  fail  the  tests,  the  entire  sub- 
array  structure  is  abandoned  as  it  would  have 
been  if  three  or  four  of  the  subarrays  shown 
in  Figure  2 had  failed. 


Figure  3:  25  Mini -Arrays 

At  this  point  the  basic  31x31  array  would  be 
divided  into  nine  subarrays  each  with  two 
planes  defined  by  its  corners  and  diagonal  as 
shown  in  Figure  4. 


Figure  4:  Nine  Subarrays 


Further  subdivisions  are  illustrated  in 
Figures  5 and  6. 


Figure  2:  Nine  Mini-Arrays 


Figure  6:  25  Mini -Arrays 


The  subdivisions  are  evident  but  the  condi- 
tions for  these  subdivisions  should  be  stated. 
If  less  than  five  of  subarrays  fail,  further 
subdivision  is  performed  as  in  Figure  5.  If 
any  of  the  planes  defined  by  the  four  mini- 
arrays of  Figure  5 occurs  the  subarray  is 
replaced  by  25  mini-arrays  as  in  Figure  6. 

No  further  reduction  occurs.  If  more  than 
five  of  the  nine  subarrays  of  Figure  4 occurs, 
the  basic  31x31  array  is  configured  into 
25  subarrays  as  in  Figure  7.  If,  then,  any 
subarray  fails  it  is  replaced  by  four  mini- 
arrays  as  in  the  left  center  of  Figure  8.  If 
any  mini-array  fails  the  test,  the  four  mini- 
arrays  will  be  replaced  by  nine  mini-arrays 
as  shown  at  right  center  of  Figure  8.  No 
additional  reduction  will  occur. 


Figure  8:  25  Subarrays  with  4 

and  9 Mini -Arrays 

Needless  to  say,  such  data  manipulation  is 
time  consuming  with  apparently  little  statis- 
tical rigor.  Grumman  proposed  to  modify  the 
GE  "Trig"  Algorithm  by  incorporating  the  Air 
Force  roughness  index  equation.  Considering 
roughness,  or  terrain  characteristics,  of 
each  31x31  word  array,  an^(,£can  be  assigned 
to  each  block  of  961  elements. 

B.  Grunman  Modification  of  Trig  Algorithm 

To  accommodate  terrain  heterogenity 
within  a 1°  x 1°  block,  the  roughness  equa- 
tion proposed  by  the  Air  Force  for  the  DRLMS 
requirements  for  the  B1  was  incorporated  and 
proposed  as  a patch  to  the  existent  software 
routine.  This  equation  will  be  used  so  that 
the  selection  ofcK  ,<£.  for  the  TRIGEN  routine 
may  be  optimized.  A statement  of  the  equa- 
tion, appears  in  Specification  CP001 -07878- 
139A  dated  3 September  1976.  From  work  done 
with  DMAAC,  probable  ranges  of <X,£  for  the 
variable^ ,£.  study  were  proposed. 

1.  Air  Force  Roughness  Equation  Statement 

Rrighness  index  is  a measure  of  terrain 
variation  and  the  equation  for  calculation 
is : 
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where  n = 31 

R = Terrain  Roughness  (Elevation  Variation) 
Index 

e = element  of  an  nxn  terrain  elevation 
i,j  matrix  whose  elements  are  spaced  in 
equal  increments  of  latitude  and  longi- 
tude. 

e = elevation  of  SW  corner  of  matrix  (meters) 

1,1 

e = elevation  of  NW  corner  of  matrix  (meters) 
n,l 

e = elevation  of  SE  corner  of  matrix  (meters) 
l.n 

e = elevation  of  NE  corner  of  matrix  (meters) 
n,n 

K = meridian  convergence  term  for  entire 
matrix 

K =fcos  (latitude  of  nth  row)  + 

L (2TTz)  

cos  (latitude  of  first  row)1  -1 


tm: t) ^ (2) 

where  Z = 1 •for  latitudes  4 1501 

= 1/2  for  latitudes  150-701 

= 1/3  for  latitudes  170-751 

= 1/4  for  latitudes  175-801 

- 1/6  for  latitudes  180-901 


2.  Interpretation  of  Roughness  Equation 

A.  Roughness  Equation 

In  a 31x  31  block,  the  elevation  of  all 
the  points  except  the  corner  points  are 
considered.  For  each  j (point  along 
the  y axis),  keeping  i constant,  the  sum 
of  the  elevations  of  the  (j+1)  st  and 
the  (j-1)  st,  terms  is  subtracted  from  2 
times  the  j1  point. 

i , j+1 
e 

1-1  ,j 

• 1 , J 

d • 

a 

c 

• 

i , j-1 


» . 4 

Fig. 9:  Identification  of  a Point  & Its  Neighbors 


For  example,  consider  the  point  i,  j- 

In  reading  equation  (1),  we  first  sum  accord- 
ing to  j.  Multiply  the  the  elevation  of  the 
point  a by  2 and  subtract  from  it  the  sum  of 
the  elevations  of  points  e and  c.  The  abso- 
lute value  of  this  difference  is  then  multi- 
plied by  a meridian  convergence  factor  (see 
eq.  2)  and  added  to  an  elevation  term  in 
which  j is  kept  constant.  This  term  is 
formed  by  multiplying  point  a by  2 and  sub- 
tracting from  it  the  sum  of  the  elevations 
of  b and  d.  Once  all  the  points  with  con- 
stant i are  considered,  i is  advanced  to  the 
next  integer  and  a similar  computation  is 
effected,  the  result  of  which  is  added  to 
the  previous  composite  sum.  There  will  be 
(n-2)^  such  calculations  comprising  the 
roughness  index  calculation.  The  meridian 
convergence  term  (k)  is  introduced  to 
standardize  the  distance  between  consecutive 
readings. 

B.  Interpretation 

The  salient  points  that  the  roughness  index 
will  quantify  for  each  area  of  topography 
subjected  to  the  test,  are  the  degree  and 
direction  that  the  slope  between  two  points 
is  changing.  For  example,  the  western 
desert  areas,  with  a constant  grade  covering 
large  areas,  will  exhibit  a RI  of  zero  the 
same  as  if  it  were  a body  of  water.  In  a 
similar  manner,  those  western  areas  in  which 
the  terrain  has  dramatic  changes  due  to 
crests  and  ridge  hills  exhibit  a significant- 
ly high  RI.  (Test  results  indicate  a value 
of  10.9  for  48/121  region).  Basically  the  RI 
is  a point-to-point  measurement  of  the  change 
rate  in  slope  between  adjacent  data  points. 

Roughness  index  will  assume  only  positive 
values.  A roughness  index  of  0 indicates 
that  the  elevation  slope  from  the  central 
point  to  each  of  its  neighbors  is  zero,  or 
constant  in  magnitude  and  direction.  A 
roughness  index  greater  than  zero  indicates 
the  slope  changes  from  the  central  point  to 
each  of  its  neighboring  points.  Let  us 
assume  that  the  points  a,b,c,d  and  e 
(Figure  9)  all  have  equal  elevations  (ELI), 
then  each  term  of  the  summation  will  be 
2( ELI ) - ( ELI +EL 1 ) or  zero.  Again,  referring 
to  Figure  9 if  the  (elevation  of  point  a - 
elevation  of  point  b)  =-  (elevation  of  point 
a - elevation  of  point  d),  and  (elevation  of 
point  a - elevation  of  point  e)  =-  (eleva- 
tion of  point  a - elevation  of  point  c),  the 
roughness  index  calculates  to  be  zero.  To 
illustrate,  consider  point  a^  of  the  contour 
map. 


i+1  ,j 
b 


I 


' 

I 

* 


Figure  10:  Contour  Map  Illustrating 
Roughness 


Although  points  b,c,d,e,  are  not  equal  in 
elevation  to  a,,  the  ilipe  of  the  terrain 
around  it  is  cinstant.  Therefore,  point  a, 
is  the  average  of  the  elevation  of  its 
equally  distant  neighbors  b,  & d,;  as  well  as 
c,  and  e,.  This  point  thereby  adds  zero  to 
tne  roughness  calculation  for  the  area. 

On  the  other  hand  point  a-  is  at  the  peak  of 
a hill,  and  is  not  the  average  elevation  of 
its  neighbors  b2>c2>d2  and  e2;  thereby,  mak- 
ing a positive  contribution  to  the  roughness 
calculation.  Since  absolute  values  are  used 
in  the  calculations,  a depression  of  equal 
magnitude  to  the  peak  at  point  a2  would  con- 
tribute an  equal  amount  to  the  roughness 
calculation. 

Regardless  of  range,  the  sample  size  for 


roughness  calculation  will  be  (n-2)  where 
n = 31.  For  short  range  at  latitude  of  40 
there  is  an  interval  of  200  feet  in  longitude 
and  300  feet  in  latitude  between  points,  in 
medium  range  a difference  of  1000  feet  in 
longitude  and  1500  feet  in  latitude  between 
points,  and  in  long  range  a difference  of 
2000  feet  in  longitude  and  3000  feet  latitude. 

3.  Results  of  Roughness  Index  Calculation 

Figures  11-16  inclusively,  illustrate  the 
results  of  the  roughness  index  calculation 
for  1.5x1. 5 min.  blocks  in  areas  of  latitude, 
longitude  combinations  of  34/77,  36/77, 
47/120,  47/121  , 48/120,  48/121,  respectively. 
East  coast  terrain  is  relatively  flat  in  com- 
parison to  that  of  the  west  coast  Reference 
to  maps  for  each  of  the  specified  areas  will 
illustrate  the  fidelity  of  the  roughness 
indices  to  the  actual  terrain  roughness^  Q 
Each  of  the  Figures  4-10  represents  a 1 x 1 
block  at  short  range.  The  specified  location 
is  that  of  the  Southwest  corner  of  the  area 
under  consideration. 


Typical  roughness  maps  can  be  used  to 
establish  ar\U,t  for  each  31x31  word 
array  (1.5x1. 5)  min.  This  will  serve 
several  purposes  among  which  will  be  the 
reduction  of  time  necessary  to  form  sub- 
arrays, when  statistical  variation  within 
a block  is  pronounced  and  also  a computa- 
tional time  reduction  for  recalculation 
when  the  memory  system  capacity  is  exceeded. 

This  effort  justified  the  need  for  modify- 
ing the  GE  "Trig"  Algorithm. 
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EW  TRAINING  IN  THE  FLIGHT  SIMULATOR 


i 

DAVID  L.  ADAMY 


Antekna,  Incorporated 
Mountain  View,  California 


INTRODUCTION 

The  development  of  sophisticated,  radar  con- 
trolled antiaircraft  weapons  and  the  development 
of  warning  and  countermeasure  devices  to  allow 
aircraft  to  fight  and  survive  against  those  weapons 
have  changed  the  whole  nature  of  air  warfare. 

Now,  warning  and  countermeasures  devices  are 
considered  an  integral  part  of  the  aircraft's 
armament,  and  training  in  the  proper  use  of 
these  devices  is  in  turn  an  integral  part  of  air- 
crew training. 

With  the  reduced  availability  of  fuel,  and  the 
resulting  drive  to  move  more  and  more  aircrew 
training  into  simulators,  there  has  been  increased 
interest  in  the  addition  of  realistic  Electonic  War- 
fare (EW)  environments  in  new  and  existing  flight 
simulators.  It  is  the  purpose  of  this  paper  to 
discuss  the  ways  in  w'hich  EW  capability  is  and 
can  be  applied  to  these  flight  simulators  and  the 
way  it  interfaces  with  the  other  simulator  sub- 
systems. 

Several  general  requirements  characterize 
the  specifications  being  generated  in  this  field: 

• The  EW  simulator  must  generate  a threat 
environment,  with  groups  of  threat  emitters 
(fixed  and  moving)  at  preprogrammed 
locations  in  the  flight  simulator's  gaming 
area. 

• Threats  must  be  triggered  by  "movement" 
of  the  simulator  into  the  vicinity  of  threats 
in  the  gaming  area. 


• The  EW  displays  in  the  simulator  cockpit 
must  accurately  reproduce  what  the 
students  would  see  and  hear  if  they  were 
really  in  the  situation  being  simulated 
(including  equipment  anomalies). 

• The  operation  of  all  receiver  and  counter- 
measure controls  must  be  accurately 
reflected  in  the  EW  environment  displayed. 

• The  EW  environment  must  be  consistent 
with  the  other  elements  of  flight  environ- 
ment presented  to  students  in  the  simulator. 

• Instructors  should  be  able  to  use  and  pro- 
gram the  EW  simulation,  using  only  the 
skills  required  to  fly  and  fight  in  the 
simulated  environment. 

While  the  techniques  described  are  applied 
particularly  to  flight  simulators,  the  same  basic 
approaches  are  equally  applicable  to  Command 
Information  Center  (CIC)  Room  Simulators  and 
Armour  Crew  Trainers. 

TWO  BASIC  APPROACHES 

Two  basic  approaches  to  the  application  of  EW 
to  flight  simulators  are  used:  Signal  Injection  and 
Emulation.  In  the  first  approach  (Signal  Injection) 
actual  RE  or  video  signals  are  injected  into  the 
input  ports  of  operational  type  receivers  or  video 
processors,  as  shown  in  figure  1.  These  signals 
are  generated  with  the  exact  characteristics  that 
the  real  signals  would  have  if  they  were  received 


Figure  1.  Signal  Injection  EW  Simulation 
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by  the  operational  equipment  under  combat  condi- 
tions - with  the  aircraft  in  the  attitude  and  loca- 
tion (relative  to  the  simulated  target)  that  the 
simulator  shows  at  that  moment.  To  achieve 
this,  a computer  controlling  the  EW  simulator 
reads  the  status  of  the  receiving  and  ECM  equip- 
ment represented  in  the  trainer,  and  the  simula- 
ted location  and  attitude  of  the  trainer  in  its 
gaming  area.  It  then  computes  the  exact  signal 
parameters  which  would  be  seen  by  the  receiver 
or  video  processor  at  that  moment.  The  EW  sim- 
ulator is  then  commanded  to  generate  those  signal 
parameters  — simulating  many  signals,  each  with 
the  proper  pulse  and  scan  characteristics,  signal 
strengths,  direction  of  arrival,  and  (if  applicable) 
RF  frequency.  This,  of  course,  causes  the 
receiver  or  video  processor  to  generate  the  proper 
output  signals  to  the  displays  which  are  presented 
to  the  student  or  students  being'  trained  in  the  flight 
simulator. 


Signal  injection  is  in  many  ways  the  simplest 
type  of  EW  Simulation  to  add  to  a simulator, 
because  the  actual  receiver  or  processing  equip- 
ment used  in  operation  is  driving  the  displays  the 
student  sees  in  front  of  him.  This  means  that  the 
display  characteristics  are  precisely  correct  — 
including  any  peculiarities  in  the  way  the  actual 
equipment  presents  its  data,  and  any  systematic 
performance  anomalies  which  the  student  will 
see  later  when  he  uses  the  same  equipment  in 
the  real  aircraft. 

In  the  second  approach  (emulation)  as  shown  in 
figure  2,  no  actual  signals  are  generated  and  no 
operational  equipment  is  used  (except  perhaps  the 
actual  displays).  Rather,  a computer  calculates 
the  data  that  would  be  present  on  the  operator's 
displays  if  the  aircraft  represented  by  the  trainer 
were  in  the  simulated  position  and  attitude  of  the 
trainer  in  its  gaming  area. 


At  first  glance,  this  seems  to  be  a more 
simple  approach,  but  now  all  of  the  little  "quirks" 
of  the  operational  equipment  displays  must  be 
reproduced  in  order  to  preclude  a negative  train- 
ing effect.  It  is  a fairly  simple  task  for  a com- 
puter to  place  alphanumeric  characters  or  lines 
on  a cathode-ray  tube  (CRT)  screen;  but  when 
these  must  jitter  around  in  different  ways  as  a 
function  of  signal  density  and  relative  location 
(as  is  the  case  in  some  real  equipment),  the  task 
becomes  more  formitable. 


SCENARIOS 


In  both  Signal  Injection  and  Emulation,  threat 
scenarios  are  stored  in  the  EW  simulator  com- 
puter. These  scenarios  comprise  large  numbers 
of  threats  located  at  various,  appropriate  positions 
throughout  the  gaming  area.  Any  of  these  may 
be  moving.  The  platform  simulator  is  "flown" 
through  this  gaming  area,  and  as  threats  are 
encountered,  they  are  displayed  to  the  student(s) 
in  the  simulator.  As  the  simulator's  "position" 
comes  closer  to  the  threat  signal  "location,  " the 
threat  signal  may  change  modes  and  the  simula- 
tor will  be  "fired  upon"  if  it  gets  into  an  appro- 
priate position  relative  to  the  threat.  The  EW 
computer  will  analyze  the  effects  of  counter- 
measures applied  at  all  stages  in  this  process, 
determining  if  the  threat  will  fire  - and  if  it 
does,  whether  or  not  it  hits  the  simulated  air- 
craft. Damage  assessment,  or  "near  miss" 
information  will  be  fed  back  to  the  platform  so 
this  can  be  applied  (if  appropriate)  to  other 
systems  in  the  simulator.  For  example,  a near 
miss  could  require  the  visuals  to  show  a burst 
or  an  undetonated  missile  flying  past  in  the 
appropriate  direction. 


Figure  2.  Emulation  EW  Simulation 
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Figure  3.  Functional  Block  Diagram  of  Radar  Warning  Receiver 


DATA  INPUT  TO  SIMULATOR 


The  EW  data  must  be  input  to  the  simulator  in 
ways  that  accurately  reflect  the  way  real  data 
enters  the  EW  equipment  in  the  simulated  aircraft. 
While  the  locations  of  the  "aircraft"  and  the 
"threats"  in  the  gaming  area  may  be  known  within 
±50  feet,  it  would  be  highly  inaccurate  to  reflect 
this  accuracy  in  the  cockpit  displays.  As  an 
example,  consider  a typical  digital  Radar  Warn- 
ing Receiver  (RWR)  with  a block  diagram  as 
shown  in  figure  3. 


This  RWR  must  determine  the  type,  mode,  and 
location  of  threat  signals  relative  to  the  aircraft 
from  the  information  it  receives  through  its  four 
antennas. 


Each  antenna  has  a gain  pattern  as  shown  in 
figure  4.  (Note  that  this  is  a cosine  pattern. ) 

The  output  amplitude  of  each  antenna  is  thus 
approximately  proportional  to  the  cosine  of  the 
angle  between  the  antenna  boresight  and  the 
signal  direction  of  arrival.  Since  the  antennas 
are  mounted  on  the  "four  corners"  of  the  air- 
craft as  shown  in  figure  5,  the  video  processor 
can  work  the  vector  equations  (figure  6)  and 
determine  the  resultant  signal  strength  and 
azimuth  of  arrival  of  each  signal  (hence  the  range 
and  bearing  to  the  threat).  Of  course,  the  proc- 
essor must  also  determine  the  signal  type  from 
the  characteristic  parameters  of  the  received 
signal. 


Figure  5.  Aircraft  Antenna  Patterns 
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Figure  4.  Typical  Warning  Receiver 
Antenna  Pattern 


Figure  6.  Components  of  Signal  Vector  in  Each  Antenna 


That's  easy  so  far,  but  there  are  some  prob- 
lems. The  first  is  that  the  signal  may  not  be 
within  the  plane  of  the  four  antennas  and  the  air- 
craft may  not  be  flying  straight  and  level.  So, 
the  computer  must  work  the  coordinate  transfor- 
mation shown  in  figure  7 to  determine  the  spheri- 
cal angle  between  the  signal  and  each  antenna 
boresight.  * Two  interesting  examples  come 
immediately  to  mind: 

o If  the  aircraft  is  upside  down,  the  RWR  dis- 
play will  be  reversed. 

o If  the  aircraft  is  in  a 90  degree  bank  on  a 
course  perpendicular  to  the  threat  signal 
vector,  all  of  the  antennas  will  see  the  same 
signal  level  — so  no  valid  display  can  be  made 
to  the  student  in  the  cockpit. 

These  are  extreme  examples,  but  they  do  serve 
to  illustrate  the  display  distortion  effects  that  must 
be  considered  if  a realistic  data  presentation  is  to 
be  generated. 

The  other  major  effect  is,  of  course,  terrain 
occlusion  of  threat  signals.  Whenever  the  simula- 
tor approaches  a threat  emitter  location,  the  com- 
puter must  check  that  the  vector  between  the 
aircraft  location  and  the  threat  emitter  location 
does  not  fall  below  the  "surface  of  the  terrain"  in 
the  gaming  area  (for  simulators  with  that  capabil- 
ity) before  a signal  is  displayed.  (See  figure  8. ) 

* The  actual  antennas  are  also  slightly  depressed 
from  horizontal  plane. 


PROGRAMMING  THE  EW  SIMULATOR 


While  the  detailed  programming  techniques 
applicable  to  individual  simulators  vary  because 
of  many  technical  and  mission  related  factors, 
there  are  some  requirements  which  they  share. 
The  most  important  of  these  is  that  the  instruc- 
tor be  able  to  program  the  EW  environment  with- 
out having  to  learn  any  skills  other  than  those 
required  to  fight  in  that  environment  and  to  teach 
others  to  fight  in  that  environment.  This  specifi- 
cally excludes  direct  knowledge  of  the  parameters 
of  signals  associated  with  individual  threats.  The 
instructor  must,  therefore,  be  able  to  specify 
threats  by  threat  type  only  (e.  g. , an  SA-X)  - and 
the  simulator  itself  must  then  set  up  the  proper 
pulse,  scan,  and  RF  frequency  for  each  of  the 
radars  associated  with  the  SA-X,  or  in  some 
other  way  establish  the  information  against  which 
the  threat  displays  will  be  generated. 


The  instructor  must  also  be  able  to  quickly 
and  easily  place  those  threats  in  the  simulator's 
gaming  area,  either  by  specifying  the  gaming  area 
coordinates  at  which  the  threats  are  to  be  placed 
or  by  placing  threats  in  some  graphic  way.  An 
example  of  a graphic  technique  is  the  movement 
of  a threat  symbol  by  use  of  a track  ball  — over  a 
map  of  the  simulator  gaming  area  shown  on  a 
CRT. 


Figure  7.  Determination  Angle  from  Signal  to  Antenna  Boresight 
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Figure  8.  Terrain  Masking  Geometry 
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Another  desirable  feature  is  the  ability  to  place 
groups  of  threats  (for  example,  a complete 
defended  airfield  or  city)  with  one  set  of  coordi- 
nates or  the  placement  of  a single  symbol  with 
the  track  ball. 

The  above  stated  programming  requirements 
assume  that  the  instructor  will  be  programming 
the  EW  simulator  by  the  placement  of  individual 
threats  or  groups  of  threats  - which  is  helpful  for 
some  types  of  training.  However,  another  (per- 
haps more  common)  situation  is  the  requirement 
to  train  the  student  to  operate  in  a "typical” 
environment,  or  in  the  environment  expected  at 
some  specific  place  on  the  earth.  In  both  of  these 
cases,  the  actual  threat  scenario  would  be  pre- 
pared for  the  instructor  by  individuals  with  current 
and  expert  knowledge  of  the  threat  environment  to 
be  simulated.  So  the  instructor  need  only  use  the 
provided  environment. 

But  whether  the  instructor  must  create  the 
environment,  or  merely  use  a preprogrammed 
environment,  he  can  still  concentrate  on  the 
instruction  and  evaluation  of  combat  skills  - not 
on  EW  skills  as  such. 


EW  TKAINING  VS.  FLIGHT  SIMULATOH 
TRAINING 

Despite  the  fact  that  the  EW  environment  is 
now  considered  an  integral  part  of  the  combat 
environment,  it  is  often  highly  desirable  to  sep- 
arate the  flying  part  of  the  simulator  from  the 
EW  part  to  conduct  different  types  of  training. 
Therefore,  it  often  becomes  yet  another  require- 
ment of  the  EW  simulator  that  it  be  usable  with 
or  without  the  flight  simulator.  The  EW  simula- 
tor shown  in  figure  9,  which  is  in  use  with  a flight 
simulator  in  Europe,  has  provision  for  the  use  of 
a parallel  set  of  warning  receiver  displays  (shown 
in  place  in  the  photograph).  Since  the  EW  simula- 
tor contains  its  own  computer,  which  stores  the 
whole  EW  environment  gaming  area,  those  front 
panel  displays  can  be  made  to  operate  just  as  they 
would  in  the  cockpit  — by  causing  the  computer  to 
"fly"  any  desired  course  through  the  gaming  area. 

But  the  real  payoff  still  comes  when  the  system 
is  used  in  its  primary  role,  to  provide  students 
with  as  close  as  possible  to  a combat  experience 
in  the  flight  simulator  by  adding  the  EW  environ- 
ment to  the  other  sensory  feedback  provided. 


Figure  9.  Typical  EW  Simulator 
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The  EA-6B  Prowler  is  the  U.S. 
Navy's  most  sophisticated  electronic 
warfare  aircraft.  Designed  to  employ 
high-power  jammers  and  other 
electronic  countermeasures  to  disrupt 
enemy  radar  and  communications  and 
thus  screen  strike  force  aircraft 
from  surface-to-air  missiles,  the 
EA-6B  also  is  used  to  protect  surface 
ships  from  radar  detection  by  enemy 
aircraft  and  from  cruise  missiles. 

Operation  of  the  complex, 
advanced  electronics  gear  on  the 
EA-6B  demands  a high  degree  of  crew 
training  and  readiness.  In  response 
to  this  requirement,  Sperry  SECOR 
of  Fairfax,  Va. , has  designed  and 
is  building  an  EA-6B  Weapons  System 
Trainer  under  a prime  contract  from 
the  Naval  Air  Systems  Command.  The 
trainer  will  help  ensure  that  U.S. 

Navy  pilots  and  electronics  counter- 
measures officers  (ECMO’s)  receive 
realistic  training  in  the  operation 
of  their  aircraft. 

The  EA-6B  trainer  which  will  be 
the  U.S.  Navy ' s largest  and  most 
technically  advanced  flight  simulation 
training  device  is  scheduled  for 
installation  at  the  Naval  Air  Station, 
Whidbey  Island,  Washington.  At 
present,  delivery  is  projected  for 
1979.  Simulation  of  the  EA-6B 
aircraft  involves  complete  aerodynamic 
modeling  of  aircraft  performance, 
including  all  flight  instruments  and 
aircraft  systems.  The  trainer  also 
provides  simulation  of  all  EW  systems 
and  on-board  avionics,  including  a 
digital  radar  landtnass  simulation.  A 
computer  generated  image  provides  an 
out- the-window  display  for  shore- 
based  anil  carrier  operations.  Full 
simulation  of  the  electromagnetic 
environment  in  which  the  aircraft  will 
operate  is  also  provided.  The 
training  device  is  designed  to  operate 
in  two  modes:  integrated  full- crew 
flight  and  tactics  operation;  and 
independent  operation  of  flight  or 
tactics  subsystems  for  specialized 
train ing . 


The  Navy  will  realize  substan- 
tial savings  in  training  costs  by 
directly  substituting  EA-6B  trainer 
flight  hours  for  actual  aircraft 
flight  hours.  These  savings  will 
accrue  from:  reduced  aircraft  flight 
hours;  lower  personnel  and  logistics 
costs  in  maintenance  and  expendables; 
accident  reduction;  extension  of 
aircraft  life  through  flight  hour 
reductions;  and  improved  and 
increased  training  capability.  The 
EA-6B  WST  is  shown  in  Figure  1 . 

The  hourly  operating  cost  of  the 
EA-6B  trainer  is  projected  to  be  10 
percent  of  the  hourly  cost  of  actual 
aircraft  operatio  1.  Based  on  the 
Navy's  plan  to  substitute  3,^400 
simulator  hours  for  1,930  hours  of 
flight  time  from  1980-1981,  a total 
savings  of  about  $3*2  million  could 
be  possible.  This  projection  is 
based  on  the  experience  of  DOD  and 
commercial  airline  users.  It  assumes 
that  the  capital  costs  of  an  EA-6B 
aircraft  (fly  away  configuration) 
and  an  EA-6B  trainer  are  roughly 
comparable.  The  potential  dollar 
savings  are  realized  primarily  from 
flight  hour  reduction  and  accident 
reduction.  Other  cost  saving  factors 
such  as  improved  training  and  lower 
maintenance  and  support  costs  offer 
potential  for  additional  savings. 

Other  training  benefits,  too, 
contribute  to  the  cost-effectiveness 
of  this  simulator.  Some  of  these 
are  difficult  to  quantify  and  can  be 
estimated  only  by  employing 
assumptions  based  on  empirical  data. 
For  example,  improved  and  increased 
training  capability  is  achieved  by 
providing  training  scenarios  that 
would  be  impractical  to  produce  using 
the  actual  aircraft.  The  simulator 
permits  training  in  emergency 
situations,  such  as  engine  fires  on 
takeoff,  which  would  not  normally  be 
induced  in  the  actual  aircraft. 

Also,  certain  combat  situations  can 
be  duplicated  in  the  simulator  with 
exact  fidelity  and  realism  which  are 
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not  possible  in  the  aircraft  except 
during  wartime  conditions.  For 
example,  the  EA-6B  trainer  provides 
the  capability  to  simulate  training 
missions  over  contested  areas  and 
enemy  territory.  Also,  the  trainer 
can  reproduce  wind,  turbulence, 
and  visibility  conditions  in  simulated 
flight  with  precisely  controlled  and 
realistic  fidelity.  These  same 
weather  conditions  may  never  be 
encountered  on  actual  training  flights 
or  would  more  likely  be  avoided  for 
reasons  of  safety. 

Major  components  of  the  EA-6B 
trainer  include:  a modified  EA-6B 
aircraft  cockpit/student  station; 
six  degree-of-f reedom  motion  simula- 
tion system;  a dusk-night  visual 
simulation  system  employing  computer 
generated  image  technology;  a three- 
position  instructor  station  for 
control  of  the  flight  and  tactics 
scenarios;  a main  simulation  computer 
complex  which  has  1.3  megabytes  of 
core  memory  and  1,000  megabytes  of 
disc  memory.  A block  diagram  of 
the  EA-6B  WST  is  shown  in  Figure  2. 

In  addition,  the  device  has  a 
maintenance  intercommunication  system, 
hydraulic  power  units  for  the  motion 
and  flight  controls,  and  other  support 
equipments  for  test  and  repair. 

The  heart  of  the  system  is  the 
computer  complex  and  associated  soft- 
ware programs.  These  integrate  the 
various  simulator  components  and 
provide  the  basis  for  operation  and 
control  of  the  device.  The  software 
development  effort  is  a major 
achievement  of  the  EA-6B  program. 

The  EA-6B  trainer  computer 
complex  consists  of  four  Central 
Processing  Units  (CPU’s),  including 
their  associated  peripherals  and  a 
four-way  shared  memory.  Each  of 
these  CPU's  controls  a distinct 
functional  subsystem:  (l)  master 
computer  for  input-output,  control  of 
instructional  features  and  synchroni- 
zation of  all  four  computers, 

(2)  flight  computer  for  aerodynamic 
calculations  and  motion  system 
control,  (3)  tactics  computer  for 
control  of  ECM  functions  and  imple- 
mentation of  tactics  and  instructional 
features,  and  ( ^ ) digital  radar  land- 


mass  simulation  computer  for  imple- 
mentation of  the  APQ-129A  radar  and 
control  of  input-output  flow  to 
cockpit  and  instructor  station 
displays.  A fifth  computer, 
independent  of  the  four,  is  used  for 
on-site  software  development.  A 
sixth  computer  is  used  to  synthesize 
the  visual  scene.  In  addition  to 
allowing  for  orderly  development  of 
software  functions,  this  architecture 
allows  for  degraded  mode  operation  in 
event  of  computer  failure.  For 
example,  should  the  tactics  computer 
malfunction,  training  for  pilots  and 
the  ECM0- 1 can  still  continue.  If 
the  master  or  flight  computer  fails, 
ECMO  2 and  3 training  will  not  be 
interrupted . 

The  EA-6B  simulator  incorporates 
the  most  modern  features  of  digital 
flight  simulation  technology.  These 
features  permit  simulation  of  the 
entire  envelope  of  aircraft  aero- 
dynamic performance  and  flying 
qualities,  including  stalls  and  spin. 
Simulation  of  the  full  range  of  engine 
performance,  all  aircraft,  auxiliary 
systems,  ground  handling,  carrier 
catapult  launch  and  recovery  wing 
fold,  ejection  seats,  G suits, 
environmental  control  systems,  and 
oxygen,  are  included  in  the  trainer. 

All  flight  instruments  and  communica- 
tions/navigation systems  are  also 
simulated.  The  device  includes 
digital  radar  landmass  simulation  of 
the  AN/APQ-129A  search  radar.  This 
system  contains  1.6  million  square 
miles  of  landmass  data  base  storage; 
terrain  and  cultural  features, 
including  shadows,  refraction  and 
earth  curvature,  far  shore  brightening, 
and  range  attenuation;  and  moving 
targets  and  emitter  occulating. 

ECM  simulation  covers  the 
AN/ALQ-99  Tactical  Jamming  System, 
AL/ALQ-92  c ommunicati ons  Jammer, 

AN/ALQ- 1 26  Defen  sive  Jamming  System 
and  AN/ALE- 129  Chaff  Dispenser. 

The  EA-6B  trainer  simultaneously 
simulates  fixed  and  moving  tactical 
emitters.  These  simulated  emitters 
are  provided  in  sufficient  quantity 
to  train  EA-6B  aircrew's  to  operate 
effectively  in  a real  world  tactical 
environment.  Various  types  of 
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emitters  are  programmable  (e.g., 
search  radar,  communications 
facilities,  fire  control  radars), 
with  each  having  the  appropriate 
electromagnetic  characteri sties  and 
signal  strengtli  location  in  the 
gaming  area. 

The  visual  presentation  is 
provided  by  a three-window,  two- 
channel  , dusk/night  computer 
generated  visual  image  simulation 
system.  Realistic  visual  presenta- 
tions of  NAS  Whidbey  Island  and  the 
USS  Nimitz  are  provided  for  training 
in  shore-based  and  carrier  operations. 
Integration  of  visual  scene  elements, 
including  lights,  beacons,  etc.  and 
proper  correlation  of  radio,  enhance 
the  realism  and  training  effectiveness 
of  the  device.  The  visual  scene 
will  display  carrier  dynamics  up  to 
sea  state  9.  A generic  countryside 
is  included,  with  horizon  and  sky  for 
cross-country  navigation  missions. 

The  hostile  area  scene  is  a 
unique  feature  of  the  visual  simu- 
lation. It  includes  special  effects 
such  as  antiaircraft  fire,  SAM 
launches  and  flares.  As  part  of  the 
visual  simulation,  developmental 
work  is  being  done  to  provide  a 
visual  scene  which  includes  a KA-6D 
tanker  aircraft  with  drogue  for 
training  in  in-flight  refueling 
operations  of  the  EA-6B  aircraft 
crew. 

With  all  these  capabilities  and 
technical  features,  the  EA-6B 
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simulator  can  have  up  to  three 
instructor  operators  to  coordinate 
and  control  the  training  scenario 
in  integrated  flight  and  tactics 
operation.  The  instructor  station 
includes  five  graphic  CRT  displays 
with  interactive  terminals.  Also, 
repeat  displays  of  the  visual  scene, 
AN/APQ-129  Radar,  digital  radar  land- 
mass  simulator  and  AN/ALQ-99  Tactical 
Jammer  are  provided  at  the  instructor 
console.  The  instructor  station  is 
designed  to  provide  maximum  flexi- 
bility in  trainer  operation.  Ten 
sets  of  initial  conditions  are 
included  to  accelerate  the  start-up 
of  training.  A number  of  pre- 
programmed demonstration  maneuvers 
are  provided.  Other  special  features 
include  twenty-minute  dynamic  replay, 
with  fly-out  capability  and  computer 
evaluation  of  trainee  performance  in 
flight  and  tactics  mode. 

The  EA-6B  weapons  system  trainer 
will  provide  better  training  at 
substantially  lower  costs  over 
traditional  training  techniques.  It 
offers  new  opportunities  to  the  EA-6B 
training  community  in  utilizing  the 
extensive  range  of  features  and 
capabilities  of  the  device.  By 
integrating  this  flight  simulator 
into  the  training  cirriculuni,  the 
U.S.  Navy's  Naval  Air  Systems  Command 
will  take  a major  step  forward  in 
providing  vitally  needed  qualified 
aircrews  for  the  EA-6B. 
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Figure  1.  EA-6B  Weapons  System  Trainer 


Figure  2.  EA-6B  Weapons  System  Trainer  System  Block  Diagram 
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ABSTRACT 

Simulation  is  of  continually  increasing 
importance  in  training.  Cost,  energy,  and 
safety  are  among  the  driving  factors.  The 
goal  of  simulator  development  is  to  provide 
increased  training  effectiveness.  This  is 
frequently  considered  to  be  almost  synony- 
mous with  increased  simulation  realism. 
Actually,  the  relationship  between  simula- 
tion realism  and  training  effectiveness  is 
far  from  simple.  Realism  itself  is  not  a 
simple  scalar  - there  are  types  of  realism, 
and  functions  of  realism  must  be  considered 
in  multidiraension  space.  This  applies  to 
cost  as  well  as  to  training  effectiveness. 
Questions  in  this  area  may  not  be  difficult 
to  answer,  once  they  are  asked.  There  is  a 
continual  danger  that  important  decisions 
will  be  made,  and  made  incorrectly,  due  to 
failure  to  ask  the  pertinent  questions. 

This  paper  considers  results  from  simulation 
experiments  and  training  activities  over  the 
years.  In  particular,  surprising  and  non- 
intuitive  results  are  examined.  The  major 
goal  is  to  illuminate  areas  of  pertinent 
questions,  although  in  some  cases  tentative 
answers  are  presented. 

INTRODUCTION 

The  title  of  this  paper  can  be  inter- 
preted in  two  ways.  It  implies  a relation- 
ship -realism  is  the  abscissa  and  effec- 
tiveness the  ordinate  and  the  nature  of  the 
functional  relationship  is  to  be  explored. 

Or  the  title  can  with  equal  validity  be 
interpreted  as  implying  an  adversary  re- 
lationship, that  realism  actually  de- 
grades training  effectiveness.  The 
following  discussion  covers  topics  under 
both  meanings. 

I was  a participant  at  a meeting  at 
Wright-Patterson  Air  Force  Base  some  time 
back.  The  topic  was  planning  of  an  elec- 
trooptical  viewing  system  (EVS)  simulation 
for  training.  Many  aspects  of  actual  EVS 
displays  were  discussed,  prompting  numerous 
questions  of  how  to  achieve  realism  in  re- 
gard to  the  area  under  discussion.  Finally, 
an  Air  Force  representative  raised  the 
issue  that,  instead  of  pondering  how  to 
achieve  realism,  we  should  ask  how  to 
achieve  training.  We  then  started  making 
some  progress. 


There's  a rather  obvious  reason  why  we 
frequently  fall  into  the  trap  of  considering 
simulation  realism  as  the  end,  rather  than 
as  the  means  to  an  end.  Given  that  effec- 
tive training  can  be  achieved  with  the  real 
system  in  the  real  environment,  it  follows 
that  a simulation  providing  100  percent 
realism  would  unquestionably  provide  effec- 
tive training.  It  does  not  follow  that  100 
percent  realism  is  necessary.  Furtner, 
given  a system  in  which  the  realism  is  short 
of  this  goal,  it  does  not  follow  that  it  is 
necessarily  desirable  to  improve  realism  in 
one  aspect,  with  no  change  in  other  areas. 

WHAT  IS  REALISM? 

We  do  not  usually  ask  this  question. 

We  know  what  realism  is,  and  consider  the 
question  of  the  degree  to  which  we  need  it, 
and  how  to  obtain  it.  Maybe  the  question 
itself  should  be  asked  more  frequently  than 
it  is. 

During  an  experiment  on  tracking  about 
the  roll  axis^)  there  were  three  modes  of 
operation.  In  one,  the  subject  saw  only  a 
front  display,  and  his  seat  was  stationary. 
In  another,  he  saw  only  the  front  display, 
but  received  roll  cues  from  valid  seat  mo- 
tion. In  a third,  the  seat  was  still,  but 
two  side  displays  were  added  to  simulate 
peripheral  vision  sensing  of  roll.  The  re- 
sults of  major  significance  to  the  current 
discussion  were  that  performance  in  modes 
two  and  three  differed  significantly  from 
mode  1,  but  the  results  for  these  two  modes 
were  very  close.  This  included  comparison 
of  both  the  amplitude  and  phase  of  the 
transfer  functions  of  operators  involved  in 
the  experiment. 

It  would  not  be  expected  that  knowledge 
of  roll  rates  and  magnitudes  obtained  by 
these  two  routes  (Figures  1 and  2)  would 
necessarily  lead  to  essentially  identical 
performance. 

Some  experiments  with  a certain  type  of 
stroke  victim  have  provided  additional  i n- 
s i ght k 2 ) . in  SOme  cases,  the  effect  of 
stroke  is  to  completely  eliminate  peripheral 
vision  - to  break  the  link  between  "Eyes" 
and  "Visual  Perception"  in  Figure  1.  The 
subject  can  only  see  the  contents  of  a nar- 
row tunnel  straight  ahead.  Nevertheless, 
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Figure  1.  Vision  Perception  of  Roll 


Figure  2.  Motion  Perception  of  Roll 


if  he  is  seated  in  a chair  and  objects  are 
moved  in  his  blind  peripheral  region,  he  re- 
acts as  though  he  senses  roll! 

This  strongly  suggests  that  the  percep- 
tion of  motion  from  objects  moving  in  the 
field  of  view,  while  it  involves  the  retina 
and  the  optic  nerve,  does  not  involve  visual 
perception  of  the  motion,  and  derivation  of 
observer  motion  knowledge  from  that.  It 
must  be  more  akin  to  the  diagram  shown  in 
Figure  3.  Now,  at  what  point  in  this  diagram 
is  "reality,"  insofar  as  it  affects  the  roll 
tracking  task.  The  subject  could  get  roll 
information  from  other  sources  — the  exper- 
imenter could  inform  him  of  roll  parameters  - 
but  this  would  not  provide  the  same  perfor- 
mance. As  a tentative  hypothesis,  if  the 
output  of  the  "Motion  Detection"  block  is 
valid,  realism  is  achieved,  independent  of 
the  input  providing  this  output. 


Figure  3.  Roll  Perception  - Composite 
Block  Diagram 

REALISM  - CHOOSE  YOUR  ROUTE 

Perception  of  distance  is  very  important 
to  many  CGI  training  activities.  This  is  an 
area  in  which  past  CGI  systems  have  not  had 
notable  success;  further,  it  is  an  area  in 
which  potential  improvement  may  be  very  dif- 
ficult and  expensive.  It  is  important  to 


understand  just  what  is  the  realism  we're 
seeking  here. 

Gibson(3)  states  that  the  mode  of  per- 
ception we  normally  think  of  first,  "Apparent 
size  of  familiar  objects. .. "is" .. . far  less 
important  and  fundamental."  It  is  "...  not 
the  kind  of  distance  perception  which  occurs 
in  the  everyday  visual  world."  His  analysis 
leads  to  the  statement,  "The  perception  of 
depth,  distance,  or  the  so-called  third  di- 
mension, is  reducible  to  the  problem  of  the 
perception  of  longitudinal  surfaces.  When 
no  surface  is  present  in  perception  because 
of  homogeneous  retinal  stimulation,  distance 
is  indeterminate." 

Interesting.  Many  years  prior  to  CGI, 
Gibson  put  his  finger  on  a major  problem,  the 
"homogeneous  retinal  stimulation"  found  on 
past  systems.  The  solution,  however,  cannot 
be  just  any  surface  nonunifcrmity.  The  dis- 
tance information  is  contained  in  the  "stimu- 
lus gradient." 

The  source  of  the  stimulus  gradient  may 
be  a texture  gradient  (the  one  normally 
thought  of  initially  when  this  problem  is 
attacked);  it  may  be  aerial  perspective;  it 
may  be  gradients  of  velocity  and  direction  in 
a dynamic  situation;  it  may  be  (for  short 
distances)  gradient  of  binocular  disparity; 
or  it  may  be  convergence  of  parallel  lines  or 
structures.  One  can  envision  a diagram  sim- 
ilar to  Figure  3,  but  with  the  various  visual 
gradients  entering,  and  a generalized  "stimu- 
lus gradient"  (on  which  decision  and  action 
are  based)  leaving. 

In  this  case,  it  may  be  valid  to  con- 
sider this  stimulus  gradient  as  the  "realism" 
we  are  after.  If  an  actual  scene  provides 
this  gradient  in  the  form  of  a texture  gradi- 
ent, and  a simulation  provides  the  same  gra- 
dient by  using  parallel  line  convergence,  it 
will  not  look  " natural,"  but  in  a training 
sense  is  realism  degraded? 

Perhaps  we  should  determine  the  approach 
which  will  give  the  most  valid  and  realistic 
stimulus  gradient,  and  deemphasize  considera- 
tions of  which  looks  most  natural. 

MATCHED  NONREALISM 

In  one  area,  there  is  no  question  as  to 
what  constitutes  realism.  If  in  the  real 
world  some  object  whose  position  provides  me 
with  information  should  move,  then  the  pho- 
tons from  that  object  reflect  the  movement 
with  a delay  limited  only  by  the  speed  of 
light.  Similarly,  if  the  dynamics  simulation 
calls  for  the  onset  of  roll  to  occur  at  a 
given  instant,  then  a realistic  motion  system 
would  provide  it  at  this  instant,  with  no 
delay. 
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A system  being  installed  by  NASA  (the 
Differential  Maneuvering  Simulator  at  Langley 
AFB)  had  a cockpit  motion  system  with  several 
degrees  of  freedom.  It  had  a camera-model 
visual  system  with  servoed  cameras.  The  dis- 
play system  included  a servoed  projector. 
Maximum  realism  is  desirable  - absolute  real- 
ism requires  zero  delay  - so  all  delays  in 
the  system  were  reduced  to  their  practical 
minimum.  The  subjective  effect  of  flying  the 
system  was  highly  unrealistic  and  unsatis- 
factory. They  then  determined  which  element 
in  the  system  had  the  maximum  irreducible 
delay,  and  they  increased  all  system  delays 
to  match  it.  The  result  was  a satisfactory 
system. 

Any  technique  has  limitations.  This 
same  approach  was  later  used  on  a different 
system  and  the  results  were  unsatisfactory. 
The  motion  platform  in  this  case  had  very 
sluggish  response.  When  the  visual  response 
was  slowed  to  match  it,  the  lag  was  so  much 
greater  than  that  of  the  aircraft  dynamics 
that  realism  was  impaired.  When  delays  were 
adjusted  without  regard  to  the  motion  system, 
the  results  were  satisfactory  - the  feedback 
from  the  visual  scene  motion  was  so  much  more 
powerful  than  that  from  the  platform  motion 
as  to  overpower  the  latter. 

WHOLE-HOG  - FINE  IF  YOU  CAN  GET  IT 

Consider  a nuclear  power  plant  simulator 
for  training  operating  personnel.  It  can 
readily  be  built  the  exact  size,  shape,  color, 
and  even  smell  of  the  actual  operating  room. 
All  meters,  indicators,  and  actuators  can  be 
those  used  in  the  actual  system,  or  can  be 
indistinguishable  from  them.  The  simulation 
software  can  readily  incorporate  algorithms 
reflecting  the  actual  plant  operation  to  the 
point  where  the  effect  of  any  actions  taken 
by  the  operators,  or  in  response  to  simulated 
changes  in  loading  or  simulated  malfunctions, 
can  be  precisely  those  which  would  result  in 
an  actual  system  under  the  conditions  being 
simulated.  Thus,  there  is  no  point  in  asking, 
"Is  this  degree  of  realism  necessary?"  It 
would  cost  more  to  determine  it  is  not  re- 
quired, than  to  go  ahead  and  give  one  hundred 
percent  realism. 

Navigational  radar  simulation  comes 
close  to  being  in  the  same  situation.  The 
resolution  is  such  that  there  are  far  fewer 
potential  distinguishable  elements  on  a dis- 
play than  for  a visual  scene,  and  the  update 
rate  leads  to  a much  greater  allowable 
processing  time  per  scene.  It  thus  becomes 
feasible  to  supply  very  close  approximations 
to  full  real  ism. 


CAN  IMPROVED  REALISM  HURT? 

We  have  already  considered  the  case  of 
various  system  delays,  where  we  are  better 
off  to  degrade  the  response  of  the  faster 
(more  realistic)  elements  so  they  match  the 
slower  ones.  This  same  type  of  effect  must 
be  considered  in  other  areas  of  realism. 

Figure  4 shows  a cylindrical  storage  tank 
simulated  by  two  techniques.  In  one  case, 
each  face  has  a tone  as  determined  by  its 
orientation  relative  to  the  illumination. 

In  the  second,  the  technique  of  shading  tone 
across  faces  is  applied  to  simulate  smooth 
curvature.  The  actual  range  of  tones  is  the 
same  on  both  - yet  it  seems  to  be  much 
greater  on  the  flat-face  version,  giving  a 
richer  set  of  cues  to  orientation  and  to  il- 
lumination direction. 


Figure  4.  Cylindrical  Tank  with  and  without 
Curvature  Simulation 

The  human  perception  system  is  exceed- 
ingly insensitive  to  gradual  continuous  changes 
of  tone  or  intensity.  This  is  ngt  a defi- 
ciency. As  pointed  out  by  Land(H)  it  is  a 
very  essential  characteristic  of  the  system 
which  enables  us  to  perceive  colors  as  con- 
stant under  wide  variations  of  lighting.  An 
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actual  tank  in  the  sun  has  gradual  gradation 
of  tone,  as  simulated  with  the  second  system. 
The  actual  scene  differs  in  another  aspect  of 
realism  - the  actual  intensities  are  several 
orders  of  magnitude  brighter  than  can  be 
achieved  in  CGI  display  systems.  If  we  could 
achieve  the  actual  brightness,  there  would  be 
no  question,  we  would  want  the  actual  shad- 
ing. Being  unable  to  match  real-scene 
brightness,  perhaps  we  do  not  want  real -scene 
shading.  We  are  not  trying  to  arrive  at  a 
validated  answer  here.  The  point  being  made 
is  that  we  should  at  least  question  whether 
to  use  maximum  available  realism  of  each  as- 
pect of  the  simulation  with  no  consideration 
of  the  range  of  effects  on  training  cues 
which  will  result. 

The  T-37  simulated  on  the  ASPT  system 
using  a large  number  of  faces  and  the  curva- 
ture algorithm,  is  extremely  effective.  An 
early  viewer  of  the  system  thought  it  was 
being  produced  separately  from  CGI  by  a model 
and  television  camera.  However,  another  Air 
Force  observer  who  flew  the  system  in  its 
early  operational  phase,  said  he  felt  that 
this  algorithm,  in  making  invisible  the  face 
edges  with  which  the  aircraft  was  defined, 
was  throwing  away  information  that  could  be 
valuable  in  training  — cues  to  replace  those 
normally  extracted  from  aircraft  skin  texture. 

DELIBERATE  DISTORTION  OF  REALISM? 

When  Navy  pilots  are  landing  on  a car- 
rier, critical  glide-path  information  is  de- 
rived from  the  "meatball,"  or  Fresnel  lens 
landing  system.  This  was  therefore  modeled 
on  the  2F90  visuals,  used  by  the  Navy  in 
Kingsville,  Texas,  for  pilot  training.  This 
CGI  system  is  an  early  one,  with  rather  lim- 
ited resolution.  At  the  distance  where 
pilots  normally  expect  to  start  extracting 
information  from  the  meatball,  they  were 
unable  to  do  so  - it  did  not  subtend  enough 
pixels  on  the  display.  The  solution  - it  was 
made  less  realistic.  Its  modeled  size  was 
made  a function  of  its  distance  from  the 
pilot  - larger  than  life  at  a distance  and 
decreasing  to  actual  size  at  close  distances. 
Result  - satisfactory  training. 

An  article  by  one  of  the  Navy's  training 
personnel  at  Kingsvil  lew)  is  instructive  in 
this  regard.  Published  in  Approach,  the 
Naval  Aviation  Safety  Review,  it  states: 
"Student  instruction  in  WEPS  stage  was  insti- 
tuted in  April  1975.  After  a short  period  of 
instruction,  it  was  evident  that  the  students 
who  had  had  several  hops  in  the  simulator 
were,  to  a significant  degree,  outperforming 
those  who  had  not."  Later,  "The  system  has 
enabled  the  instructor  to  provide  the  student 
aviator  with  a high  degree  of  realism..." 

Note  - that  important  word,  realism.  The 
article  includes  several  pictures  showing  a 


bomb  run  over  the  Navy-generated  data  base 
used  for  this  training.  The  ground  is  cov- 
ered with  orthogonal  sets  of  stripes  re- 
sembling a checkerboard.  Real  ism?  The 
author  obviously  did  not  mean  pictorial 
realism,  but  realism  as  it  applied  to  his 
task,  to  train  the  pilots. 

:)EALISM  TYPES  AND  COMBINATIONS 

Consider  the  visual  scene  realism  re- 
quired for  a couple  of  types  of  training. 

Landing  an  aircraft  requires  that  the 
proper  glide  slope  be  maintained  on  ap- 
proach; that  descent  be  arrested  at  the 
proper  point  on  the  glide  slope;  and  that 
rate  of  vertical  descent  be  held  very  close 
to  zero  as  airspeed  decreases,  culminating 
in  the  goal  of  touchdown  at  the  desired 
point,  at  the  proper  airspeed,  and  with  as 
close  as  possible  to  zero  rate  of  vertical 
descent.  Mastery  of  this  sequence  in  a 
simulator  requires  close  to  perfect  realism. 
The  spatial  and  perspective  validity  of  the 
simulated  scene  must  be  exact.  The  response 
of  the  visual  scene  to  the  trainee's  air- 
craft control  cannot  be  approximate  — it 
must  be  precise.  Every  student  pilot  has 
many  times  executed  a perfect  landing,  meet- 
ing all  above  requirements,  but  one  foot 
above  the  runway  surface  — and  then  dropped 
and  bounced,  hoping  no  one  was  watching. 

This  illustrates  the  degree  of  realism 
required. 

Consider  next  the  training  of  navigators 
for  nap-of-the-earth  helicopter  flying.  In 
this  type  of  operation,  the  aircraft  avoid 
roads,  rivers,  and  settlements.  These  would 
greatly  ease  the  navigator's  task,  but  they 
would  also  subject  the  helicopter  to  sight- 
ing by  the  enemy.  They  fly  low;  below  hills, 
even  below  the  tops  of  trees.  Thus,  the 
navigator  cannot  look  down  on  a panorama  and 
locate  standard  map  features.  The  naviga- 
tors must  base  their  decisions  on  extremely 
subtle  visual  cues  — the  lay-of-the-land  as 
expected  from  map  contour  information,  a 
glimpse  of  a pond  through  the  trees,  etc. 

To  train  navigators  for  this  task  essen- 
tially perfect  realism  is  needed  — the 
smallest  variations  in  elevation  of  the 
actual  terrain  must  be  faithfully  reflected 
in  the  simulated  visual  scene.  Slight  color 
variations  can  be  very  significant.  The 
scene  must  look  exactly  as  the  terrain 
would  from  any  viewpoint  and  attitude. 

Here  are  two  training  requirements,  each 
requires  as  close  as  possible  to  perfect 
realism,  yet  quite  inexpensive  solutions  to 
each  are  in  use. 

CGI  systems  producing  low-detail  scenes 
which  have  ful ly  valid  dynamic  and  interactive 


fidelity  are  providing  effective  training  in 
tasks  such  as  takeoff  and  landiny  and  weapons 
delivery.  These  include  calligraphic  dis- 
plays and  limited-capability  solid-object 
display  systems. 

Effective  nap-of-the-earth  helicopter 
navigator  training  is  being  provided  by  sys- 
tems based  on  viewing  motion  pictures  taken 
during  flights  through  the  training  area, 
with  increasingly  difficult  assignments  as 
the  training  proceeds.  There  is  no  inter- 
active capability  - the  scene  proceeds  in- 
dependently of  the  trainee's  actions  - but 
none  is  needed.  The  fully  valid  pictorial 
fidelity  needed  is  provided. 

COST  VERSUS  REALISM 

To  provide  a pictorial  indication  of  the 
relationship  between  realism  and  cost,  it  is 
necessary  to  use  at  least  a two-dimensional 
plot.  Figure  5 shows  the  nature  of  the  rela- 
tionship. The  point  at  the  lower  right, 
designated  "A,"  has  limited  pictorial  realism 
but  full  dynamic  and  interactive  realism  - 
this  is  the  inexpensive  region  occupied  by 
the  flight  trainer  referred  to  above.  The 
point  near  the  upper  left  designated  "B"  has 
no  interactive  capability  but  excellent  pic- 
torial realism  - it  is  the  inexpensive  region 
occupied  by  the  nap-of-the-earth  navigation 
training  system. 

The  system  complexity,  and  thus  cost, 
rises  steeply  when  a system  must  provide  both 
types  of  realism  simultaneously.  The  current 
trend  of  systems,  particularly  for  the  mili- 
tary, is  along  the  right  edge  of  the  figure, 
moving  back  in  the  direction  of  continually 
increasing  image  fidelity.  There  is  no  ques- 
tion that  training  in  tasks  such  us  target 
identi t ication  and  weapon  damage  assessment 
requires  pictorial  realism  comparable  to  the 
navigation  task  discussed  above.  If  it  re- 
quires this  simultaneously  with  highly  dynamic 
an^4  interactive  realism,  then  the  require- 
ments fall  in  the  right  rear  of  the  plot.  In 
all  such  cases,  however,  it  is  important  to 
consider  whether  the  secondary  type  of  real- 
ism can  have  relaxed  requirements,  as  this 
can  drastically  lower  system  complexity. 

REALISM  CAN  BE  HARMFUL  TO  YOUR  HEALTH 

This  one  still  hurts  when  I think  about 
it.  It  was  back  in  1969  or  1970.  We  had  our 
real-time  laboratory  system  working,  and 
driving  a wide-screen  display  in  front  of  a 
simulated  cockpit.  We  were  expecting  an  im- 
portant group  of  vistors  - potential  users 
and  customers.  We  were  modeling  the  world 
they  would  fly  in. 

CGI  scenes  had  been  criticized  for  look- 
ing too  stylized,  too  cartoonish,  unnatural. 


Figure  5.  Visual  Scene  Simulation  Cost 
Versus  Fidelity 

We  did  all  we  could  to  cure  that.  Fields 
were  made  a variety  of  shapes  and  sizes  - 
not  just  square.  Roads  were  laid  with  vary- 
ing directions,  not  in  parallel  patters' 

We  did  all  we  could  to  make  things  look  nat- 
ural -to  add  realism. 

Black  Monday  arrived.  The  vistors  flew 
the  system.  Monday  aftermon  we  had  a de- 
briefing. Every  pilot  who  had  flown  the 
system  had  praise  for  many  espects  of  the 
simulation,  but  stated  he  was  "unable  to 
maintain  his  glide  path."  No  one  knew  why, 
but  all  agreed  that  unless  we  could  deter- 
mine why,  and  fix  the  problem,  this  type  of 
system  might  be  worthless  for  training. 

Then  a couple  of  them  made  the  comment  which 
provided  the  critical  clue.  They  said  they 
were  more  successful  in  maintaining  the  de- 
sired glide  path  when  shooting  a landing  ori 
the  carrier,  where  there  were  no  surface 
cues  at  all.  This  indicated  our  land  scene 
was  providing  counter-cues. 

We  redid  the  terrain.  We  made  all  fields 
square,  and  roads  parallel.  We  took  all  the 
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naturalness  out  of  it.  Two  days  later, 
Wednesday,  we  had  the  modified  scene  up  and 
running.  Many  of  the  visitors  were  still  at 
the  plant,  and  were  persuaded  to  fly  it 
again.  They  were  amazed.  A typical  comment, 
"I  don't  know  what  you  did  to  it,  but  you 
sure  fixed  it." 

In  retrospect,  we  know  what  we  did.  We 
did  not  have  many  scene  edges  then.  In 
Wednesday's  scene,  those  we  had  were  devoted 
to  providing  Gibson's  "stimulus  gradient" 
derived  from  the  parallelism  of  scene  fea- 
tures, rather  than  destroying  this  cue  in  an 
attempt  to  achieve  a more  real  appearance. 

SOMETIMES  IT  DOES  NOT  SEEM  TO  MATTER 


In  an  interesting  experiment  several 
years  ago(°),  a simulator  was  available  in 
which  realism  could  be  varied  in  three  sig- 
nificant aspects.  The  scene  could  be  mono- 
chrome, or  full  color  — color  is  more 
realistic.  The  resolution  could  be  varied  — 
finer  resolution  is  more  realistic.  The  dis- 
play couiJ  be  collimated,  or  a real  image  on 
a screen  — collimated  is  more  realistic. 

The  experiment  was  to  determine  the  ef- 
fect on  performance  of  these  variations  in 
realism.  The  subjects  were  experienced 
pilots  - not  students.  The  measured  perfor- 
mance parameters  included  rms  deviation  from 
correct  glide  path,  deviation  from  planned 
touchdown  point,  and  vertical  velocity  at 
touchdown.  The  greatest  surprise  in  the  re- 
sults was  that  the  full  range  of  variation  in 
the  factors  affecting  realism  produced  only 
minor  perturbations  in  the  measured 
performance. 

Lateral  localizer  error,  for  example, 
ranged  from  an  rms  value  of  0.7  degree  with 
screen  display  in  color,  to  an  rms  value  of 
0.9  degree  with  the  collimated  display  in 
monochrome. 

Mean  touchdown  rate  of  descent  with  col- 
limated optics  was  3.58  feet  per  second  with 
standard  deviation  of  1.52  feet  per  second, 
and  for  the  real -image  system  the  rate  was 
4.33  feet  per  second  and  deviation  1.98  feet 
per  second. 

The  only  statistically  significant  re- 
sult reported  as  due  to  decreased  resolution 
was  in  increase  from  0.33  degree  to  0.35  de- 
gree in  rms  glide  slope  error. 

There  was  no  difference  in  results  of 
statistical  significance  between  monochrome 
and  color. 

Gibson  provides  some  insight  into  re- 
sults such  as  this”).  "The  stimulus-variable 
within  the  retinal  image  to  which  a property 


of  visual  space  corresponds  need  be  only  a 
correlate  of  that  property,  not  a copy  of 
it."  "The  correspondence  between  the  world 
and  the  optical  image  need  not  be  that  be- 
tween a thing  and  its  copy;  it  need  only  be 
that  between  a material  quality  and  its 
correlate." 

Now,  referring  to  the  experiment,  con- 
sider that  the  pilot  accepts  whatever  in- 
formation is  provided  on  the  world  in  which 
he  is  flying.  He  subjects  this  to  a corre- 
lation process  which  results  in  an  internal 
model  of  this  world.  He  then  bases  his  per- 
formance on  this  internal  model.  Thus,  as 
long  as  the  simulation  results  in  the  same 
output  from  this  correlation  process,  per- 
formance will  be  the  same,  regardless  of 
variations  in  details  of  the  simulation. 

This  would  explain  the  results  of  the 
experiment. 

MY  GOSH!  ARE  YOU  AGAINST  REALISM? 

Heck  no!  Not  only  does  increased  real- 
ism make  the  scene  look  much  prettier,  it 
significantly  reduces  distracting  effects 
which  remind  a trainee  he  is  really  in  a 
simulator.  Further,  improvements  in  real- 
ism, properly  used,  will  inevitably  improve 
training  effectiveness.  The  point  of  this 
discussion  is  that  this  is  an  extremely  com- 
plex area  of  consideration,  not  at  all  the 
simple,  black  and  white  situation  it  is  fre- 
quently considered.  Full  consideration  of 
all  areas  is  necessary  both  in  deciding  on 
efforts  to  increase  available  realism,  and 
in  deciding  just  how  best  to  use  this  real- 
ism when  it  is  available. 

We  are  all  for  realism.  We  were  the 
first  to  deliver  solid-object  CGI,  as  com- 
pared with  earlier  wire-figure  simulations. 
We  developed  the  extremely  realistic  fog 
and  haze  simulation  now  found  in  all  CGI 
systems.  We  did  not  come  up  with  the  con- 
cept embodied  in  the  curvature  simulation 
algorithm,  but  we  were  the  first  to  have  it 
operational  in  a real-time  system.  We  have 
continually  improved  realism  by  implementing 
measures  to  eliminate  edge  steps  and  other 
distracting  quantization  effects.  Variable 
sunshading,  aerial  perspective,  moving 
models,  bomb-burst  indication  - there's  a 
long  list.  But,  you  might  ask  as  in  the 
punch  line  of  the  old  joke  - "What  have 
you  done  for  me  lately?" 

Figure  6,  produced  on  our  laboratory 
scene  generation  system,  illustrates  some  of 
the  new  realism-enhancement  features  cur- 
rently being  designed  into  real-time  systems. 
Circular,  spherical,  and  ellipsoidal  fea- 
tures are  used  to  make  clouds,  trees,  water 
towers,  etc.  These  are  quite  significant. 
Development  of  efficient  techniques  for 
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Figure  6.  Scene  with  Ellipsoidal  Features  and  Texture 


producing  these  broke  CGI  free  of  its  earlier 
straight  line  constraint.  In  many  applica- 
tions, a single  circular  feature  can  replace 
many  edges. 

The  relation  between  objects  and  their 
shadows  provides  very  sensitive  motion  and 
orientation  cues.  This  figure  shows  circular 
features  used  for  shadow  simulation.  The 
real-time  shadow  generator  currently  under 
development  will  use  both  circular  features 
and  edges,  as  appropriate,  for  shadow 
simulation. 

On  the  ground,  this  figure  shows  the 
newly  developed  surface-map  texture.  It  com- 
bines two  sources  of  stimulus  gradient  infor- 
mation — texture  gradient  and  paral  lei -1  ine 
gradient.  Before-and-after  scenes  show  it  to 
be  tremendously  effective  in  orienting  all 
other  scene  features  properly  relative  to  one 
another. 

Of  course,  systems  currently  being  de- 
signed will  also  incorporate  quantitative  in- 
creases in  the  edges,  faces,  moving  objects, 
and  other  elements  traditionally  applied  to 
generate  visual  cues  for  training. 

WHAT'S  NEXT? 

We  are  looking  into  several  new  areas 
now,  that  we  are  not  even  talking  about  yet. 
In  all  cases,  we  are  attempting  to  apply  the 


principles  discussed  in  this  paper  to  assure 
that  we  do  not  go  after  realism  for  the  sake 
of  realism,  but  that  vie  achieve  advances 
where  they  are  most  needed,  and  where  the  re- 
sults can  most  cost  effectively  contribute  to 
the  training  missions  with  which  the  systems 
will  be  used. 
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TRAINING  EVALUATION  OF  THE  HITMORE  SYSTEM 


EDWARD  AYRAL  and  MAJOR  J.  CHANDLER 

Imaging  Systems  Division 
Government  Systems 
Fairchild  Camera  & Instrument  Corp. 


INTRODUCTION 

Recent  advances  in  the  state-of-the-art  in  video  cameras 
employing  all  solid-state  charge  coupled  device  (CCD) 
technology  have  prompted  a re-evaluation  of  current 
training  techniques  and  devices.  This  paper  discusses  the 
evaluation  of  this  new  technology  in  a training  environ- 
ment. 

The  Fairchild  TOW  Helicopter  Installed  Television  Monitor 
& Recorder  (HITMORE)  was  developed  to  provide  a capa- 
bility for  real-time  monitoring  and  assessment  of  gunner 
performance  and  immediate  post-mission  playback  and 
analysis  of  gunner-aim  point  during  live  or  simulated  firings 
of  the  TOW  Weapon  System. 

In  this  helicopter  application,  the  gunner,  located  in  the 
front  seat  of  the  AN-IS  TOW  COBRA,  utilizes  a stabilized 
Telescopic  Sight  Unit  (TSU)  with  which  he  can  detect  and 
accurately  track  a target.  As  an  aid  to  gunner  training,  and 
for  effectiveness  evaluation,  provisions  for  a 16mm  film 
gun  camera  form  a part  of  the  TSU.  Training  benefits  of 
this  film  record  are  minimal  because  of  the  several  day  de- 
lay between  exposure  and  screening  of  the  film  due  to  film 
processing  requirements.  Additionally,  light  level  variations 
limit  the  usefulness  of  film  cameras.  Another  training  aid, 
the  Gunner  Accuracy  Control  Panel  (GACP)  displays  azi- 
muth and  elevation  gunner  errors  to  the  instructor -pilot 
(IP)  in  the  second  seat  of  the  helicopter,  but  this  system  is 
usable  only  with  specially  conditioned  targets. 

HITMORE  was  developed  to  overcome  these  shortcomings 
by  providing  the  instructor-pilot  with  a real-time  image  of 
gunner's  field-of-view,  including  the  TSU  reticle,  on  a high 
brightness  video  monitor.  In  a training  exercise,  the  IP  can 
observe  and  verbally  correct  the  way  in  which  the  gunner 
sights  on  the  target  and  maintains  position  from  initial  ac- 
quisition to  impact.  Tracking  errors  and  jitter  in  azimuth 
or  elevation  can  be  observed  dynamically  and  corrected  in- 
stantaneously. 

For  immediate  detailed  review  of  gunner  performance 
upon  return  to  base,  the  on  board  video  tape  recorder 
(VTR)  can  be  utilized.  The  video  tape  cassette,  removed 
upon  landing,  can  be  replayed  on  a VTR  and  displayed  on 
a video  monitor  for  performance  review,  assessment  and 
correction;  this  procedure  can  take  place  much  more  ra- 
pidly than  in  the  case  of  16mm  film  and  therefore  repre- 
sents a significant  training  aid  improvement  for  the  IP  and 
student.  Relative  advantages  of  the  HITMORE  are  sum- 
marized in  Table  1 . 


TABLE  1 

VIDEO  TRAINING  BENEFITS 

■ No  Target  Conditioning  Required 

■ Real-Time  IP  Observation/Verbal  Queing 

■ Immediate  VTR  Playback  on  Landing 

■ 40  Simulated  Firings  on  a Single  Cassette 

■ Reusable  Tape  Cuts  Cost  Over  Film 

■ Short  Term  Record  to  Demonstrate  Improvements 

■ Modular  Growth 

■ Operation  Value 
- RECCE 

— Damage  Assessment 
— Landing  Air 

HITMORE  CAMERA  SYSTEM  DESIGN 

The  Fairchild  HITMORE  is  made  possible  by  the  recent 
availability  of  small  rugged  low-light  level  TV  cameras, 
high  brightness  monitors,  and  video  tape  recorders  rugged- 
ized  for  use  in  the  helicopter  environment.  The  smallest, 
most  rugged  and  reliable  TV  cameras  available  employ  a 
solid-state  imaging  device  rather  than  a vidicon  tube.  Fair- 
child  has  developed  solid-state  charge  coupled  device 
(CCD)  area  imaging  arrays  and  cameras  which  are  ideally 
suited  to  this  requirement.  Operation  at  the  low  light  levels 
available  from  the  TSU  beam  splitter  is  possible  because  of 
the  superior  CCD  sensitivity. 

The  CCD  camera  is  mounted  in  a special  bracket  which 
maintains  the  same  TSU  interface  as  the  previous  film 
camera  and  receives  information  from  an  optical  beam- 
splitter within  the  sight.  The  dynamic  range  and  AGC 
characteristics  of  the  CCD  camera  permit  effective  opera- 
tion over  a wide  range  of  scene  brightness  without  expo- 
sure control.  This  provides  a state-of-the-art  system  with 
high  reliability  at  a modest  cost. 

The  HITMORE  camera  is  depicted  in  Figure  1.  The  CCD 
camera  output  feeds  both  a video  tape  recorder  and  high 
brightness  monitor.  HITMORE  Power  is  switched  manual 
ly  via  a remote  switching  and  control  panel  conveniently 
located  for  operation  by  the  IP.  Automatic  tape  recorder 
shut-off  is  provided  with  a manual  override  as  a tape  saving 
feature  and  assures  that  the  recorder  is  not  inadvertently 
left  in  a RECORD  mode  for  extended  periods.  A warning 
light  indicates  end  of  tape. 
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FIELD  EVALUATION 


FIGURE  1 THE  HITMORE  CAMERA 


The  CCD  television  camera  was  designed  to  assure  that 
mechanical  and  optical  interface  is  maintained  without 
modification  to  the  TSU.  Locating  the  small  CCD  format 
(7.2mm  diagonal)  in  the  16mm  film  image  place  (12.6mm 
diagonal)  reduces  the  displaced  field-of-view  to  67%.  This 
provides  improved  target  resolution  and  image  magnifi- 
cation and  enhances  the  instructor's  ability  to  assess  gun- 
ner accuracy  at  the  projected  time  of  target  intercept. 


VIDEO  TAPE  RECORDER 
AND  MONITOR  INSTALLATION 

A High  Brightness  Video  Monitor  is  mounted  on  the  top  of 
the  gunners  seatback  for  viewing  by  the  instructor  pilot 
(Figure  2).  The  small  size  of  the  display  serves  to  minimize 
obstruction  of  the  IP's  forward  view  in  this  location,  and 
the  increased  visibility  offered  by  use  of  the  television 
camera  more  than  compensates  for  the  area  blocked  by  the 
monitor.  The  video  tape  recorder  also  depicted  in  Figure  2 
is  located  in  the  space  immediately  behind  the  pilot's  seat. 


FIGURE  2 HITMORE  IN  AH  IS  HELICOPTER 


In  May  1978  three  (3 ) HITMORE  systems  were  installed 
at  Fort  Campbell,  Kentucky  and  in  July  1978,  another 
three  (3)  systems  were  installed  in  Germany.  Installations 
were  accomplished  at  both  sites  just  prior  to  extensive 
maneuvers  and  exercises  with  the  TOW  missile.  Before  the 
exercises  commenced,  a field  questionnaire  (Appendix  1) 
was  distributed  for  completion  by  both  procurement  and 
user  type  personnel.  Although  this  paper  is  being  written 
prior  to  receipt  and  tabulation  of  all  questionnaires,  a 
number  of  salient  evaluation  factors  have  surfaced: 

1)  TRAINING  EFFECTIVITY 

Training  effectivity  was  defined  as  the  probability  of  a 
first  round  hit  after  training  with  HITMORE  as  compared 
to  the  probability  of  a first  round  hit  with  no  training.  It 
was  estimated  that  this  probability  improved  from  approx 
imately  75%  to  better  than  95%  with  the  use  ofHITMORE. 

2)  TACTICAL  VALUE 

Instructor  pilots  and  gunners  determined  that  targets  could 
be  acquired  several  seconds  faster  when  the  HITMORE 
system  was  utilized. 

3)  AUTOMATIC  LIGHT  CONTROL  (ALC) 

A special  camera  with  an  auto  iris  was  supplied  prior  to 
field  evaluation  in  Germany  to  determine  the  ALC  systems 
value.  The  basic  system,  without  ALC,  requires  that 
neutral  density  filters  be  changed  to  accommodate  varying 
light  conditions.  It  was  discovered,  however,  that  the 
dynamic  range  of  the  camera  was  sufficient  to  provide  ac- 
ceptable images  throughout  all  the  lighting  conditions 
encountered  during  training  without  ever  changing  the 
neutral  density  filter.  Therefore,  it  would  seem  that  ALC 
could  add  complexity  with  little  system  value. 

4)  NIGHT  COMPAT ABILITY 

During  night  exercises  it  was  determined  that  the  control 
box  instrument  lights  were  not  compatible  with  use  of  the 
AN/PVS-5  night  vision  goggles.  The  dimmer  provided 
could  not  reduce  the  light  output  sufficiently  to  prevent 
pilot  blinding.  The  alternative  suggested  was  use  of  a mech- 
anical iris  in  front  of  each  light.  The  optimum  iris  setting 
must  be  determined  empirically.  This  solution,  if  satis- 
factory, will  be  incorporated  in  subsequent  HITMORE 
systems. 


CONCLUSIONS 

■ The  HITMORE  system  has  positive  training  value 

■ The  tactical  value  of  HITMORE  should  be  further 
investigated 

■ Some  minor  design  modifications  are  advisable 
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APPENDIX  1 


FIELD 

QUESTIONNAIRE 


on 


The  Military  Training  Potential 
of  the  Helicopter  Installed 
Television  Monitor  and  Recorder 
(HITMORE) 


Prepared  by 


Captain  Jerry  L.  Chandler 
Training  Analysis  and  Design 
Directorate  of  Training  Developments 
US  Army  Aviation  Center 
Fort  Rucker,  Alabama  36362 


June  1978 


IL 


USER  DATA:  Please  complete  the  following  informational  summary  before 
proceeding  to  the  next  page. 

- Unit: 

- Location : 

- Date  completed:  

- Ihity  Position: 

- Cobra/TOW  Experience  (Years) 

- Previous  Experience  with  similar  system:  Yes  No 

(If  yes  was  checked  complete  the  following) 

Where  

When  (Dates) 

Type  System 

- Experience  with  HITMORE  (hours)  

- Please  state  briefly  how  the  HITMORE  System  will  affect  your 
particular  job  if  introduced  as  a permanent  AH-1  Cobra/TOW  training 
aid. 


HITMORE  User  Evaluation  Questionnaire 


1.0  Purpose : 

This  instrument  is  specifically  tailored  to  collect  information 
relevant  to  the  operational  and  functional  characteristics  of  the 
Helicopter  Installed  Television  Monitor  and  Recorder  (HITMORE) 
System. 


2.0  Scope : 

User  responses  are  keyed  to  those  operational  and  functional  areas 
which  will  be  most  frequently  encountered  by  the  AH-1  Cobra/TOW 
crew  and  supporting  AVUM/AVIM  personnel.  Comments  on  the  Technical 
Specifications  of  the  system  are  not  specifically  called  for,  but 
would  be  welcomed  nonetheless,  to  the  extent  deemed  appropriate 
by  the  respondent.  However,  should  recommendations  include  informa- 
tion which  most  probably  will  result  in  system  modification  and/or 
redesign,  it  is  requested  that  supporting  drawings,  photographs,  etc; 
be  submitted  with  a narrative  description  of  the  recommended  changes. 


3.0  Uses: 

The  information  received  from  this  questionnaire  will  be  used  in 
future  development/procurement/evaluation  decisions  on  the  HITMORE 
System.  Therefore,  it  is  imperative  that  each  respondent  remain  as 
objective  as  possible  in  evaluating  the  system. 


4.0  General  Instructions r 

4.1  This  questionnaire  is  divided  into  four  sections.  Section  I is 
an  evaluation  of  the  contractors  performance  as  relates  to  the  re- 
quirements outlined  in  contract  DAAK50-78-C-0003(P6D) . Section  II 
rates  the  performance  of  each  component  of  the  HITMORE  System  both 
individually  and  collectively.  The  Third  Section  is  an  assessment 
of  the  military  utility  of  the  HITMORE  System  as  viewed  by  both  the 
operating  crew  and  support  personnel . Section  IV  is  reserved  for 
open-ended  discussion  of  future  design  and/or  applications  of  the 
HITMORE  System.  There  is  no  restriction  to  the  length  or  scope  of 
the  comments  the  respondent  may  wish  to  include  in  this  section.  In 
addition  to  written  comments;  drawings,  photographs,  sketches,  etc., 
are  encouraged. 
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4.2.1  Section  I will  be  completed  by  the  contracting  officer/ 
assistant  contracting  officer  following  receipt  and 
acceptance  inspection  of  HITMORE  System(s) . 

4.2.2  Sections  II,  III,  and  IV  will  be  completed  by  user  personnel 
during  and  after  system  installation/evaluation. 


i 
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SECTION  I 


Contractor  Performance 

Yes  No 


Systems  were  developed  IAW  contract 
specifications? 


Functional  checks  met  established 
operational  criteria? 


Systems  design  met  stated 
Government  standards? 


Systems  were  delivered  on  schedule? 


Contractor  met  all  performance 
requirements  of  contract  agreement? 


System  was  developed  within  initial 
cost  estimate? 


Contractor  was  responsive  to 
government's  needs  throughout 
system  development? 


Contractor  fulfilled  field  support 
requirements  IAW  contractual 
agreement? 


Contractor  has  prepared  adequate 
documentation  to  support  reliability, 
availability,  maintainability 
predictions  for  each  component  of 
HITMORE  System? 


Remarks 
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SECTION  II 


Equipment  Performance 

M 

-Remarks. 

- Each  component  functional 
(Bench) ? 

as  expected  „ 

— 

- System  function  was  as  expected  (Bench)? 

- System  installation  was  accomplished 
without  incident  or  unnecessary  delay? 

- System  functional  as  expected  (Aircraft)? 

- System  overall  performance  met  expected 
standards? 


System  is  within  AVUM  level  capability? 


“ System  must  be  maintained  at  AVIM 
level ? 


System  installation/operation  instruction 
manual  is  easily  understood  by  user 
personnel? 


System  installation/operation  instruction 
manual  adequately  describes  HITMORE 
System  operation/ function? 
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SECTION  III 


-Military  Utility 

Yes  No 


Hie  HITMORE  System  is  an  adequate 
TOW  training  aid  in  its  current 
configuration? 


The  TV  monitor  component  of  the 
HITMORE  System  is  considered  necessary? 


The  TV  monitor  component  should  be 
retained  and  moved  to  an  alternate 
location  to  permit  use  of  the  M-73 
sight  assy  for  mixed  load? 


Relocation  of  the  TV  monitor 
component  to  the  left  side  of  the 
pilot's  instrument  panel  glare 
shield  would  not  adversely  degrade 
the  pilot's  field  of  view? 


The  HITMORE  System  would  be  an 
equally  effective  TOW  training  aid 
without  the  TV  monitor  component? 


The  TV  monitor  component  displays 
sufficient  brightness  and  contrast 
to  facilitate  pilot  viewing  of  TSU 
sighting  in  direct  sunlight? 


The  TV  monitor  component  brightness 
and  contrast  controls  are  well 
located  and  easily  accessible  to 
the  pilot  during  flight? 

Futare  development  of  the  TV 
monitor  component  should  include 
compatability  with  night  vision 
goggles? 


Remarks 
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Section  III,  continued 

Yes  No 

- The  HITMORE  System  control  panel 
is  adequate  in  its  current 
configuration? 


- The  function  lights  on  the  system 
control  panel  are  excessively  bright 
during  night  operations  and  should 
incorporate  a dimming  rheostadt? 


- The  colors  assigned  to  each  of  the 
control  panel  functions  are 
acceptable  and  should  not  be  changed? 


- The  automatic  record  and  manual 
override  features  of  the  control 
panel  are  both  necessary? 


- Both  the  30  and  60  second  record 
selections  on  the  control  panel  are 
considered  necessary? 

- All  switches  on  the  control  panel  are 
readily  accessible  and  easily 
operated  by  the  pilot  during  flight? 

- The  internal  playback  feature  of  the 
video  tape  component  should  be  within 
the  capability  of  the  pilot  for 
immediate  film  replay  during  flight? 


- The  location  and  mounting  of  the 
video  tape  component  are  acceptable 
from  the  standpoint  of  operation 
and  pilot  convenience? 


.3 


Remarks 


308 


Section  III,  continued 

- The  "voice-over"  recording/playback 
capability  are  necessary  features  of 
the  video  tape  component? 


- The  capability  for  transferring  control 
of  the  video  tape  component  from  the 
pilot  to  the  gunner  station  should  be 
incorporated  into  the  pilots  HITMORE 
System  control  panel? 


Yes 


No 


Remarks 
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OPTIMIZING  MEDIA  SELECTION 

OR.  J.  OUN  CAMPBELL 
and 

DR.  JOHN  HUGHES 
Courseware,  Inc. 


This  paper  presents  a procedure  for 
optimizing  the  selection  of  study  session 
and  device  session  media  up  to  but  not  in- 
cluding life  cycle  costing.  The  procedure 
is  presented  as  a detailed  sketch,  rather 
than  a complete  guide  to  the  process,  since 
the  process  must  be  adjusted  to  fit  individ- 
ual situations. 

Media  selection  involves  three  compo- 
nents: (a)  determination  of  media  require- 
ments for  each  objective,  (b)  analysis  of 
the  requirements  for  individual  objectives 
into  optimized  mixes  of  training  media,  and 
(c)  selection  of  a final  mix  based  upon  cost 
and  availability.  This  paper  presents  a 
number  of  steps  to  accomplish  the  first  two 
components.  The  steps  are  flowcharted  in 
Figure  1. 

Objectives  follow  different  paths  after 
they  are  sorted  into  study  session  and  device 
session  (hands-on)  types.  Study  session 


objectives  are  evaluated  for  special  re- 
quirements like  audio  or  motion  portrayal, 
and  all  media  which  satisfy  those  require- 
ments are  listed.  A set  of  alternate  study 
session  media  plans  is  then  prepared,  with 
each  plan  giving  priority  to  some  media  over 
others.  For  each  plan,  in  turn,  the  highest 
priority  medium  which  satisfies  the  special 
requirements  for  each  objective  Is  selected. 
The  number  of  objectives  for  each  medium  is 
then  totalled  for  each  plan,  and  any  adjust- 
ments caused  by  overemphasizing  one  medium 
are  made.  Device  session  objectives  are 
classified  by  their  instructional  presenta- 
tion/response requirements  and  then  by  the 
particular  hardware/software  requirements 
of  the  device  being  trained.  Objectives  are 
consolidated  to  yield  descriptions  of  train- 
ers. The  outcome  of  the  present  procedure 
is  a list  of  training  devices  which  is  opti- 
mized for  the  particular  set  of  objectives 
being  analyzed. 
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Each  step  will  now  be  described  in  more 
detail . 

STEP  1 

Sort  objectives  into  study  session  or 
device  session  types 

The  first  major  activity  in  the  media 
selection  process  (Step  1)  is  to  sort  objec- 
tives into  two  groups:  those  which  are 
study  session  type  objectives,  and  those 
which  should  be  presented  in  a "hands-on" 
device  session  exercise.  Objectives  are 
sorted  because  a different  process  is 
applied  to  the  objective,  depending  upon 
which  category  it  is  in. 

Study  session  objectives  are  those  like 
stating  a fact  or  steps  of  a procedure, 
which  require  self-study  materials,  or  do 
not  require  major  devices.  Study  session 
objectives  may  require  additional  materials 
(e.g.,  logs,  charts,  calculators,  or  pro- 
tractors). So  long  as  these  additional 
materials  can  be  stored  and  issued  from  the 
learning  center,  the  objectives  should  be 
considered  of  the  study  session  type.  Ob- 
jectives which  require  a particular  spatial 
layout,  tactile  sensations,  or  movement,  or 
which  require  a device  which  cannot  easily 
be  used  in  a learning  center  are  classified 
as  hands-on. 

Study  Session  Media 

STEP  2 

Designate  objectives  as  example-set 
or  non-example  set  types 

The  next  step  for  study  session  ob- 
jectives (Step  2)  is  to  determine  whether 
the  objective  is  an  example-set  objective 
or  a non-example  set  objective.  Example-set 
objectives  are  those  which  require  multiple 
examples  in  order  to  present  the  full  range 
of  possibilites  for  the  objective.  A con- 
cept in  which  the  student  must  classify  an 
item,  or  a complicated  rule  which  requires 
the  student  to  use  many  types  of  inputs  are 
example-set  objectives.  On  the  other  hand, 
facts  or  procedures  which  are  always  per- 
formed the  same  way  may  be  taught  by  presen- 
tation of  a single  statement  of  the  fact  or 
procedure  (perhaps  with  a clarifying  figure 
or  illustration)  and  do  not  require  an 
example  set. 

STEP  3 

Identfy  special  requirements  for  Study 
Session  objectives 

In  Step  3,  special  requirements  are 
noted.  These  may  include  audio,  color, 
visual  motion,  manipulation  of  two  dimen- 


sional simulated  hardware,  direct  instructor 
evaluation,  automated  right-wrong  scoring, 
and  high  rate  of  revision. 

Objectives  which  require  an  audio  com- 
ponent are  those  based  upon  aural  discrimi- 
nation or  identification  such  as  recognizing 
a particular  missile  warning  tone  or  voice 
communication.  Audio,  like  other  special 
requirements,  should  be  specified  only  if  it 
is  essential  totfie  performance  of  the  ob- 
jective. 

"Nice-to-have"  characteristics  should  not  be 
specified  at  this  point.  Color  should  be 
specified  as  an  instructional  requirement 
only  when  the  learner  must  make  a discrimi- 
nation or  identification  based  upon  color. 
Visual  motion  (e.g.,  film  or  videotape)  is 
required  when  the  student  must  evaluate  or 
identify  a motion  change  as  such.  For 
example,  rapid  changes  in  gauge  readings  or 
in  targets  which  are  to  be  tracked  require 
visual  motion. 

The  objective  must  also  be  evaluated  in 
Step  3 for  the  special  response  requirement 
of  manipulating  hardware  controls  as  repre- 
sented on  a flat  screen  or  picture.  That 
is,  a student  may  be  required  to  push  buttons, 
read  dials  or  flip  switches  where  the  fidel- 
ity of  the  system  is  less  important  than  the 
location  and  sequence  of  the  actions.  For 
example,  setting  up  a complex  piece  of 
equipment  may  be  taught  using  a computer- 
driven  cathode-ray  tube  to  simulate  the 
face  of  the  equipment  and  a light  pen  for 
the  student  to  indicate  switch  settings  and 
button  pushes. 

Special  response  detection  and  evalua- 
tion requirements  must  also  be  specified  in 
Step  3.  Occasionally  an  objective  will 
require  direct  instructor  evaluation.  Auto- 
mated right-wrong  scoring  should  be  used 
where  the  student  requires  immediate  feed- 
back about  an  action  which  can  be  relatively 
easily  evaluated.  For  example,  the  student 
may  be  required  to  solve  a series  of  problems 
where  partial  answers  must  be  evaluated  be- 
fore the  student  progresses  to  the  next 
portion  of  the  problem. 

Expected  rate  of  revision  must  also  be 
specified  since  media  have  differing  ease 
of  revision.  For  example,  very  frequently 
changing  material  may  best  be  delivered  by 
lectures  or  on  a computer-assisted  instruc- 
tion system  (CAI),  where  a single  change  to 
the  disk  automatically  produces  changed 
copies  each  time  the  material  is  used.  More 
stable  material  might  be  presented  by  work- 
book, where  the  material  may  be  edited, 
typed,  proofread,  pasted-up,  and  printed. 
Slide  tapes  and  video  tapes  are  usually  the 
least  appropriate  media  for  rapidly  changing 
material . 
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STEP  4 


List  acceptable  media  for  each  Study 
Session  objective 

Once  the  special  requirements  of  each 
objective  have  been  identified,  they  are 
compared  to  the  capabilities  of  already 
known  and  existing  study  session  presentation 
media  (Step  3).  The  result  of  the  compar- 
ison is  a list  of  all  study  session  media 
which  can  be  used  for  presentation  of  the 
instruction  for  that  objective.  One  of  the 
outputs  of  this  matching  of  objective  re- 
quirements to  media  capabilities  may  be  the 
identification  of  some  objectives  for  which 
no  existing  presentation  media  is  acceptable. 
New  types  of  media  or  combinations  of  media 
must  be  identified  to  handle  the  require- 
ments of  these  objectives,  for  which  the 
precise  characteristics  have  been  delineated 
in  steps  2 and  3. 


Figure  2 presents  a matrix  of  special 
characteristics  by  study  session  media. 
Figure  2 is  used  by  placing  a check  beside 
each  special  required  characteristic  for 
the  objective,  then  locating  each  media 
column  which  includes  all  special  required 
characteristics.  In  this  way  all  candidate 
media  will  be  identified  which  can  support 
the  particular  objective. 

STEP  5 

Rank  media  in  alternate  plans 

A result  of  Step  4 will  be  a number  of 
media  which  are  all  capable  of  presenting 
instruction  for  a given  objective.  In  order 
to  choose  between  media,  it  is  necessary  to 
rank  the  different  media.  In  step  5,  a 
number  of  plans  for  utilizing  media  are 
developed  based  on  alternate  media  rankings. 
Figure  3 presents  a sample  media  ranking. 
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NOTES  TO  MATRIX 


1 Direct  instructor  observation  can  be  available  in  these  media.  However,  this  implies 
an  additional  resource  requirement  not  normally  associated  with  these  media 

2 While  an  automated  response  detection  and  evaluation  system  is  not  normally 
present  with  these  media,  it  is  possible  to  add  this  capability.  However,  this  addition 
in  capability  requires  an  increase  in  the  resources  needed  to  suppori  the  medium 

3.  This  cepebility  depends  on  the  cherectertsitcs  of  the  model  used. 

4.  Whether  or  not  a CAI  system  has  these  display  capabilities  is  highly  dependent  on  the 
characteristics  of  the  CAI  system.  As  a general  rule  these  characteristics  are  NOT 
normally  available. 

Figure  2.  Matrix  of  Special  Characteristics  by  study  session  media 
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STEP  6 


V 


Study  Session  Madia  Ranking  A 

CEILING 

MEDIUM 

PERCENTAGE 

1. 

Workbook 

(70%) 

2. 

Workbook  with  color  photographs 

(70%) 

3. 

Audlotapa  with  workshaat 

(50%) 

4. 

Slldaa  and  workshaat 

(50%) 

5. 

Random  accass  slldaa  and  workshaat 

(50%) 

6. 

Slldaa,  audlotapa,  and  workshaat 

(50%) 

7. 

Vldaotapa  and  workshaat 

(25%) 

An  automatad  tast-scortng  and  recording  davlca 
will  also  ba  included  In  this  plan  to  handle 

record- keeping  requirements 

Figure  3.  Sample  Media  Ranking 


Each  medium  which  is  a candidate  is 
listed  in  priority,  so  that  an  objective 
which  can  be  satisfied  by  several  media  will 
be  assigned  to  the  medium  of  highest  prior- 
ity. In  addition,  an  arbitrary  ceiling 
percentage  is  established  for  each  medium 
within  the  mix  so  that  one  medium  (like 
workbook)  is  not  assigned  too  large  a per- 
centage of  the  objectives  to  achieve  high 
effect. 

The  rankings  should  be  based  upon 
present  availability  of  media,  estimated 
cost,  need  for  central  control,  abilities 
and  preferences  of  the  target  audience  and 
the  rapidity  and  extent  of  revisions.  If 
the  client  has  a variety  of  good  equipment 
available  (e.g.,  word  processors,  drafting 
tables,  stat  camera,  printing  press,  slide 
facilities,  TV  studio),  one  of  the  alter- 
nate media  mixes  should  maximize  use  of 
present  equipment.  One  mix  should  minimize 
estimated  front-end  costs.  The  costing 
must  be  rough  at  this  stage  but  can  involve 
acquisition  of  production  equipment,  or  for 
example,  an  expensive  CAI  system.  Another 
mix  may  emphasize  centralized  control  using 
CAI  and  minimize  lectures  which  tend  quickly 
to  become  non-standardized.  Yet  another 
mix  may  optimize  ease  of  revision  using 
lectures  and  CAI  while  severely  limiting 
videotapes. 

The  developer  must  also  consider  student 
ability  and  effect.  For  example,  students 
with  a low-reading  level  should  not  be  given 
large  amounts  of  workbooks.  The  developer 
must  consider  how  students  will  feel  if  a 
large  portion  of  the  study  session  instruc- 
tion is  in  workbook  o,  any  one  medium,  and 
limit  the  total  percentagt  for  each  medium 
in  the  mix  (this  applies  primarily  to  the 
high-priority  media  in  each  mix). 


Determine  study  session  media  for  each 
objective  for  each  plan 

Once  the  media  mixes  have  been  estab- 
lished, they  can  be  used  along  with  the  list 
of  acceptable  study  session  media  for  each 
objective  to  determine  the  study  session 
medium,  since  each  mix  lists  the  media  by 
priority. 

STEP  7 

Total  number  of  objectives  in  each 
medium  for  each  plan- 

Step  7 totals  the  number  of  objectives 
presented  in  each  medium  for  each  media  plan. 

If  the  total  for  any  medium  is  greater 
than  its  allotted  percentage,  the  developer 
must  decide  whether  to  accept  the  overage, 
to  add  effective  elements  (like  cartoons  to 
workbooks),  or  to  reassign  objectives.  Re- 
assignment can  be  based,  for  example,  on 
selecting  lessons  which  are  likely  to  be 
revised  frequently  and  presenting  them  in 
lectures,  or  selectina  lessons  which  do  not 
require  color  but  would  be  enhanced  by  it 
and  presenting  them  on  slide/tape. 

Choice  between  alternative  plans  can 
be  based  upon  management  factors  like  cost, 
need  to  accomplish  rapid  revisions  and  need 
for  central  control.  Each  media  mix  in- 
cludes every  study  session  objective,  with  a 
justification  "or  the  objective's  assignment 
to  a given  medium. 

Device  Session  Media 

The  procedure  used  to  specify  device 
session  devices  is  different  from  that  used 
for  study  session  media,  because  device 
sessions  must  usually  model  a specific 
device  and,  therefore,  the  devices  must  be  de- 
signed from  scratch  rather  than  using  off- 
the-shelf  systems  like  video  tapes  or  slide 
tapes.  Study  session  materials  usually  are 
relatively  independent  of  the  particular 
equipment  being  trained:  A slide  tape  can 
present  the  procedures  used  to  preflight  any 
number  of  a set  of  pieces  of  gear,  while  a 
device  session  requires  much  greater  fidel- 
ity to  the  Individual  pieces  of  equipment. 

STEP  8 

Determine  instructional  presentation 
requirements  for  each  "device  session"  ob- 
jective. 

Those  objectives  which  were  identified 
in  Step  1 as  requiring  a hands-on  exercise 
are  examined  during  Step  8 to  determine 
their  instructional  presentation 
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requirements.  The  worksheet  shown  In  Figure  4 
can  be  used  to  record  the  Information  for  each 
objective.  The  types  of  devices  should  be 
specified  In  advance.  Possible  device  types 
are: 

A.  Schematic  representation  (e.g.,  of 
system  flow)  with  line  drawings,  photos, 
or  visual  motion  but  not  in  proper 
spatial  orientation. 

B.  Schematic  representation  with  tactile 
sensation,  manipulation  of  hardware, 
and  real-time  interaction. 

C.  Two-dimensional  static  display  with  the 
same  spatial  layout  as  actual  equipment. 


J.  Simulator  with  motion  base  and  perform- 
ance playback. 

STEP  9 

Classify  each  "device  session"  objec- 
tive in  terms  of  its  hardware/software 
system  capability  requirements^ 

In  addition  to  classifying  each  hands- 
on  objective  in  terms  of  its  instructional 
presentation  requirements,  it  is  also  nec- 
essary that  each  be  classified  in  terms  of 
its  hardware/ software  system  capability 
requirements.  This  is  important  since 
hands-on  devices  are  often  expensive  to 
develop  and  costly  to  operate.  For  this 
reason,  the  training  devices  must  possess 
the  right  combination  of  instructional  and 
hardware/software  capabilities  to  allow  for 
maximum  utilization  of  the  devices,  accomp- 
lishment of  all  objectives,  and  minimum 
development  and  operating  costs.  The  pro- 
cedure presented  in  this  step  is  not  algor- 
ithmic, in  that  it  results  in  one  correct 
answer  which  satisfies  all  of  the  above 
criteria.  It  is  analytic  in  that  it  allows 
for  systematically  examining  the  variables 
which  go  into  determining  the  "optimal 
solution."  The  final  grouping  of  re- 
quirements and  identification  of  a trainer 
suite  should  be  based  on  the  results  of  this 
procedure  together  with  knowledge  of  cost 
and  availability  of  present  trainers.  For 
example,  it  may  be  less  expensive  to  modify 
an  existing  complex  device  or  to  stimulate 
the  actual  equipment  than  to  design  and 
build  a less  complex  device.  Moreover,  it 
may  be  less  expensive  to  design  and  build 
two  copies  of  a complex  device  than  to  de- 
sign and  build  two  separate  devices,  one 
which  is  complex  and  another  which  is 
simpler. 

In  order  to  determine  hardware  and 
software  requirements,  a table  is  constructed 
which  lists  the  course  objectives  down  the 
side  and  hardware/software  systems  and  sub- 
systems across  the  top.  Figure  5 presents 
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Figure  4.  Matrix  of  Characteristics  of 
Device  Sessions  Types 


an  example  of  a small  portion  of  such  a 
table.  The  proper  column  headers  can  be 
identified  using  the  following  procedure: 

First,  for  each  training  track,  each 
hardware/software  component  directly  util- 
ized, inspected,  or  serviced  in  a hands-on 
mode  is  listed.  Components  directly  util- 
ized or  manipulated  by  the  person  are  includ- 
ed as  well  as  integrated  components  at  the 
level  of  detail  normally  utilized  on  the 
job.  For  example,  integrated  panels  rather 
than  single  switches  are  included  for  a 
pilot. 

Following  the  list  of  single  components 
above,  all  hardware  combinations  cormonly 
utilized  simultaneously  can  be  listed.  An 
example  for  a pilot  of  two  hardware  systems 
conmonly  used  simultaneously  is  the  Head-Up 
Display  (HUD)  and  the  Flight  Control  System. 
First,  list  all  pairs,  then  all  triples 
(e.g.,  the  HUD,  Flight  Control  System,  and 
communication  panels),  then  all  four  types, 
etc.,  until  all  common  combinations  have 
been  enumerated.  If  several  people  work  as 
a team,  include  all  equipment  required  by 
the  team. 

Once  this  table  has  been  constructed  for 
all  training  tracks,  the  hardware/software 
system  requirements  can  be  identified.  To 
do  this,  for  each  objective  find  the  simple 
column  which  represents  the  mix  of  hardware 
and  software  systems  and  subsystems  needed 
to  adequately  perform  the  objective.  If  no 
column  corresponds  to  the  necessary  hardware 
and  software  requirements,  then  a new 
column  should  be  added  to  the  table.  Once 
the  appropriate  column  has  been  found,  the 
letter  of  the  acceptable  device  presentation/ 
response  type  is  entered  in  that  box.  For 
example,  in  Figure  5,  objective  1.2.2  re- 
quires the  Flight  Control  System  (FCS).  The 
letter  of  the  device  presentation/response 
type  (Type  A)  is  written  in  the  intersection 
of  objective  1.2.2  by  FCS.  Only  one  column 
is  used  for  each  objective. 
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Figure  5.  Sample  table  of  objectives  by  hardware/software  components 


STEP  10 

Consolidate  instructional  presentation 
requirements  for  each  trainer  hardware/ 
software  systems-requl rements  category. 

The  objectives  must  be  grouped  in  some 
systematic  way  in  order  to  arrive  at  an 
optimal  set  of  trainer  descriptions.  The 
first  step  in  this  consolidation  process  is 
to  determine  the  presentation/response  type 
which  is  adequate  for  all  objectives  with 
each  given  hardware/software  system  (i.e., 
columns  in  Figure  5). 


most  complex  devices,  and  all  objectives  in 
the  column  are  assigned  to  the  complex  de- 
vice. This  is  done  since  the  device  must 
be  constructed  in  any  event  to  satisfy  an 
objective,  and  it  should  be  used  to  its 
maximum. 

STEP  11 

Assign  objectives  to  training  devices 
and  determine  final  trainer  configurations. 

a.  Identify  hardware/software  system 
requirements  of  most  complex  trainer. 


Usually  complex  devices  can  accommodate 
the  presentation/response  requirements  of 
objectives  assigned  to  simpler  devices. 
Therefore,  each  hardware/software  column 
is  scanned  to  find  the  one  or  possibly  two 


In  relative  terms,  the  training  device  which 
is  most  expensive  to  develop  and  operate  is 
the  device  which  is  most  critical  to  use 
properly.  Therefore,  the  hardware  and 
software  requirements  of  the  most  complex 
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device  should  be  determined  first  to  allow 
maximum  flexibility  in  the  assignment  of 
objectives  to  that  device.  This  is  done  by 
examining  the  training  device  requirements 
table  which  has  been  consolidated  with  a 
column,  and  identifying  the  column  or  columns 
which  contain  the  largest  number  of  hardware/ 
software  (H/S)  system  requirements. 

The  instructional  presentation/response 
requirements  for  that  set  of  H/S  system 
requirements  was  determined  in  step  9 and 
consolidated  in  step  10. 

These  two  sets  of  requirements  serve  as 
the  basis  for  characterizing  the  capabilities 
of  the  most  complex  training  device.  Using 
this  information,  a tentative  description  of 
the  training  device  is  developed  to  outline 
the  proposed  capabilities  and  features  of 
the  device. 

b.  Estimate  trainer  time  available  for 
each  student  on  the  most  complex 
trainer. 

Having  tentatively  specified  the  re- 
quired characteristics  of  the  most  complex 
trainer,  it  is  possible  to  identify  hands-on 
objectives  which  can  be  accomplished  by  using 
that  trainer.  However,  prior  to  beginning 
this,  a question  which  must  first  be  answered 
is:  How  many  objectives  can  be  assigned  to 
the  trainer  to  fill  (but  not  overfill)  the 
time  available  on  it?  A formula  to  compute 
time  available  for  each  trainer  has  been 
developed.  Essentially  it  multiplies  number 
of  days  in  the  training  program  by  hours 
available  per  day,  then  divides  by  number  of 
students.  Corrections  are  made  for  antici- 
pated downtime  and  proportion  of  day  used. 

c.  Assign  objectives  to  the  trainer 
until  all  available  trainer  time 
has  been  used  up. 

Once  steps  a and  b have  been  completed, 
it  is  possible  to  begin  making  a fairly 
firm  assignment  of  objectives  to  the  device. 
To  ensure  that  the  most  appropriate  objec- 
tives are  assigned  to  the  device,  the  follow- 
ing priorities  may  be  used. 


PRIORITY  I 

Objectives  which  require  the  full  range 
of  both  the  instructional  presentation  and 
response  (P/R)  capabilities,  and  the  H/S 
system  capabilities  of  the  trainer  device. 


PRIORITY  II 

Objectives  which  will  require  the  full 
range  of  the  H/S  system  capabilities  but 
less  than  the  full  range  of  P/R  capabilities. 


PRIORITY  III 

Objectives  which  will  require  somewhat 
less  than  the  full  range  of  H/S  system 
capabilities  but  the  full  range  of  the  P/R 
capabilities. 

PRIORITY  IV 

Objectives  which  will  require  less  than 
the  full  range  of  both  the  P/R  capabilities 
and  the  H/S  system  capabilities. 

All  objectives  in  Priority  I class  should 
be  assigned  prior  to  assigning  any  objectives 
in  Priority  II  class.  When  choosing  between 
objectives  within  priority  classes,  those 
objectives  which  come  closest  to  using  the 
full  range  of  capabilities  should  be  assigned 
first.  When  choosing  between  objectives  in 
Priority  IV  class,  the  H/S  system  require- 
ments should  be  the  controlling  factor  for 
assigning  ojbectives. 

d.  Repeat  trainer  requirements  deter- 
mination and  objectives  assignment 
process  for  each  required  trainer. 

Once  the  requirements  have  been  deter- 
mined for  the  most  complex  trainer  and  ob- 
jectives have  been  assigned  to  it,  the  pro- 
cess outlined  in  steps  a,  b,  and  c can  be 
repeated  with  the  remaining  hands-on  objec- 
tives for  the  next  most  complicated  trainer. 
The  process  should  be  repeated  again  and 
again  until  all  trainers  have  been  identified 
and  all  objectives  have  been  assigned. 

At  this  point,  it  is  useful  to  examine 
the  entire  set  of  trainer  assignments.  The 
procedure  recommended  in  this  approach  will 
result  in  all  trainers,  except  the  last  of 
each  type,  having  enough  objectives  assigned 
to  them  to  fill  all  available  time.  For 
this  reason,  the  trainer  assignments  should 
be  examined  to  see  if  some  minor  shifts  can 
be  made  in  the  assignments  to  equalize  the 
usage  levels. 

e.  Develop  detailed  description  of 
each  trainer. 

Once  satisfactory  assignment  of  ob- 
jectives to  trainers  has  been  reached,  it 
is  necessary  to  prepare  a more  detailed 
specification  of  the  required  capabilities 
of  each  training  device.  The  instructional 
presentation  capabilities  and  the  hardware/ 
software  capabilities  should  be  specified 
in  detail.  These  descriptions  should  be 
sufficiently  complete  so  that  they  can 
serve  as  input  to  the  development  of  the 
trainer  functional  specifications. 

Following  this  process,  it  is  necessary 
to  develop  a detailed  set  of  functional 
specifications  and  detailed  costing  and 
support-requirements  estimates  for  each  pro- 
posed trainer.  These  steps  need  to  be  worked 
out  with  manufacturers  of  the  equipment. 
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INTRODUCTION 


Advanced  Technology  Systems  (ATS),  wel- 
comes this  opportunity  to  describe  our  ad- 
vanced computer  generated,  full-color,  day/ 
dusk/night  Simulator  Visual  System  which  we 
call  COMPUTROL*.  We  believe  that  the  system 
represents  a genuine  breakthrough  in  the 
state  of  the  art,  in  that  the  level  of  picture 
detail  far  exceeds  that  of  currently  available 
systems. 

Among  the  noteworthy  features  of  the  ATS 
system  are: 

Its  ability  to  produce  a full-color,  day/ 
dusk/night  picture  including  blue  lights.  A 
minimum  of  10,000  colored  point  light  sources 
can  be  generated  and  displayed. 

Its  ease  of  generating  a new  picture.  A 
new  airport  can  be  programmed  in  one  working 
day. 

Its  ability  to  display  30,000  edges.  Ex- 
pansion to  100K  edges  possible. 

Its  ability  to  drive  additional,  inde- 
pendently controlled  displays. 

Realistic  special  effects  such  as  horizon 
glow,  variable  cloud  cover,  variable  visibil- 
ity, correct  sun  angle,  moving  traffic  and 
lights  whose  intensity  varies  in  slant  range. 

No  blooming  of  lights  at  the  end  of  run- 
way. 

As  will  be  discussed  later,  the  technol- 
ogy to  produce  a high-definition  picture  with 
full -color,  smoothly  rounded  curves  and  in- 
finite shading  has  been  developed  with  illus- 
trative photographs  to  prove  the  point.  Color 
shading  has  also  been  developed  for  transition 
of  one  color  to  another  without  abrupt  or 
noticeable  effects. 

BACKGROUND 

For  those  not  acquainted  with  ATS,  please 
permit  a slight  digression.  We  are  an  operat- 
ing division  of  The  Austin  Company,  an  organi- 
zation of  engineers,  designers,  and  builders 
established  in  1878  and  operating  coast-to- 
* Registered  Trademark 


coast  in  the  U.S.  and  12  international  offices. 

ATS  was  organized  as  the  Special  Devices 
Division  to  handle  classified  research  and 
development  projects  and  has  been  devoted  to 
the  advancement  of  the  state  of  the  art  of 
visual  simulation  since  its  founding  in  1943. 

Our  credits  include: 

A torpedo  and  rocket  attack  trainer  which 
features  the  first  spherical-domed  visual  sys- 
tem of  the  simulated  world  environment  in 
which  the  trainee  maneuvered  (12  units  built). 

Every  Submarine  Periscope  Training  System 
used  by  the  U.S.  Navy.  Many  now  include  our 
40:1  diffraction  limited  zoom  lens  which  revo- 
lutionized the  Submarine  Periscope  View  Simu- 
lator performance. 

EARLY  CGI  DEVELOPMENTS 

Our  first  Computer  Image  Generator  was  a 
nighttime  system  for  the  simulation  of  harbor 
navigation.  ATS  designed  and  developed  a 
color,  nighttime,  dynamic  CGI  system  which 
provided  a 360°  view  capability  with  simulated 
own  ship  speeds  up  to  60  knots.  The  system 
had  300  point  lights,  some  of  which  were  hood- 
ed to  permit  viewing  to  a restricted  F0V.  The 
simulated  buoy  lights  blink  at  controllable 
rates  which  permits  the  maneuver  into  and  out 
of  a simulated  harbor. 

This  early  system  also  simulated  aircraft 
flight  providing  maneuverability  for  takeoff, 
landing,  and  airborne  maneuvers.  Recognizing 
the  inadequacies  of  current  daytime  CGI  tech- 
niques, ATS  embarked  upon  the  development  of 
the  COMPUTROL  Day/Dusk/Night  Image  Display  and 
Control  System. 

The  goal  of  this  developemnt  was  increas- 
ed capability  for  the  real-time  manipulation 
of  perspective  views  of  three-dimensional  ob- 
jects. The  resulting  system  design  will  be 
capable  of  generating  enhanced  detail  for 
terrain,  cultural  features,  and  moving  target 
models  while  also  displaying  such  special 
effects  as  contrails,  weapon  impacts,  and 
transparencies. 


321 


Upon  becoming  acquainted  with  the  accom- 
plishments of  the  Human  Resources  Research  Or- 
ganization's (HumRRO)  CHARGE  System,  the  two 
organizations  decided  to  combine  technical  and 
financial  resources  for  the  fulfillment  of 
common  goals. 

The  CHARGE  (Color  Half-tone  ARea  Graphics 
Environment)  System,  part  of  a HumRRO  study 
conducted  in  1971  and  1972,  addressed  Computer 
Aided  Instruction  (CAI)  techniques.  The  pur- 
pose was  to  develop  specifications  for  a total 
CAI  system  with  components  that  included  hard- 
ware, software,  lesson  plans,  and  instruction- 
al decision  models. 

Image  generation  techniques  were  first 
modeled  in  software  so  that  alternative  algor- 
ithms and  hardware  architecture  could  be 
studied,  simulated,  and  verified  before  com- 
mitting image  generation  functions  to  hardware. 
The  resultant  hardware/software  design  ex- 
ceeded expectations,  rivaling  in  sophistica- 
tion and  performance  any  CGI  system  then  on 
the  market. 

As  a result  of  a continuing  effort  in  the 
study  of  image  generation  techniques,  ATS/ 
HumRRO  has  evolved  the  basis  for  the  current 
advanced  design.  While  committing  certain 
image  generation  functions  to  hardware  to 
achieve  an  order  of  magnitude  increase  in 
processing  capability,  the  system  still  re- 
tains immense  versatility  both  in  hardware 
and  software  flexibility.  The  main  advantage 
is  that  it  can  handle  more  edges  in  real-time 
at  less  cost.  Any  additional  or  new  image 
generation  algorithms  can  be  incorporated 
since  algorithms  are  not  completely  frozen 
into  special-purpose  hardware. 

APPLICATION  SCENERIO 

Although  a CGI  system  can  be  tailored  to 
match  a variety  of  visual  requirements,  this 
paper  relates  to  aircraft  operations  rather 
than  other  vehicles'  use.  We  have,  therefore, 
related  our  discussion  to  aircraft  visual 
simulation. 

The  COMPUTROL  visual  system  will  present 
a CGI  scene  to  the  pilot  which  reproduces  the 
outside  world  viewed  through  the  windshield 
during  ground  handling,  traffic  pattern,  and 
low-level  maneuvers  as  well  as  high-altitude 
flight  for  both  day  and  nighttime  training. 

With  the  exception  of  a flight  data  inter- 
face with  the  simulator  host  computer,  the 
visual  system  is  a self-contained  digital 
image  generator  with  bulk  storage  of  speci- 
fied geographical  data  bases.  An  off-line 
data  base  modeling  capability  is  included. 


The  visual  system  will  allow  realistic 
training  in  all  maneuvers,  specifically: 

Takeoff  with  critical  engine  failure. 

ILS  approach  with  engine  failure. 

Landing  with  engine  failure. 

Rejected  landing. 

Rejected  takeoff. 

Takeoff  with  crosswind. 

Approach  and  landing  with  crosswind. 

Instrument  takeoff. 

Nonprecision  approach. 

The  visual  scene  will  follow  all  motions 
computed  by  the  simulator  and,  therefore,  can 
visually  reproduce  any  maneuver  of  which  the 
simulator  is  capable. 

DESIGN 

The  visual  system  covered  by  this  paper 
consists  of  the  following  components: 

Digital  computer  Simulator  interface 

Image  processing  hardware 

Cockpit  displays 

Gaming  area/storage  retrieval  system 

Support  Equipment  (SE)  and  Built-in  Test 

Gaming  area  modeling  equipment 

The  visual  computer  consists  of  a special- 
purpose  digital  processor  and  related  equip- 
ment. Insertion  of  a new  gaming  area  into 
core  requires  less  than  ten  seconds. 

BASIC  SYSTEM  HARDWARE  CONFIGURATION 

The  general  configuration  of  the  COMPU- 
TROL system  consists  of  essentially  three 
components:  Image  Generator,  Memory/Decoder 
Units,  and  Disc  Storage  Units. 

The  image  generator  performs  in  real- 
time all  functions  required  to  generate  a per- 
spective view  from  a compiled  world  whose  un- 
specified parameters  are  derived  from  the 
user's  input  device  or  from  another  computer. 

The  memory/decoder  units  receive  from  the 
image  generator  the  edges  representing  the 
selected  two-dimensional  projection  of  three- 
dimensional  objects  and  buffers  and  decodes 
the  edges  for  display  on  the  monitor  screen. 
One  or  more  memory/decoders  can  be  assigned  to 
a color  montior. 

The  disc  storage  units  store  the  3D  de- 
finition of  objects  and  surfaces  comprising 
the  data  base  gaming  area.  This  representa- 
tion of  the  real  world  is  logically  segmented 
into  related  object  sets  so  that  a "user"  may 
roam  through  a world,  not  all  of  which  can  be 
held  in  the  image  generator's  main  memory  at 
once. 
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The  image  generator  consists  of  two 
parallel  processors:  the  projection  processor 
and  the  visible  surface  processor.  Each  pro- 
cessor consists  of  custom  designed  high-speed 
controllers  and  arithmetic  units  which  com- 
municate with  a specially  designed  high-speed 
CPU  and  main  memory. 


The  image  generation  capability  of  COMPU- 
TROL  will: 

Display  up  to  30K  edges  in  any  single 
channel  or  throughout  a wide  angle  FOV.  Edges 
may  be  utilized  to  model  point  light  sources, 
ellipsoidal  surfaces,  etc. 


CPU 


Initially  the  architecture  of  the  CPU 
was  designed  with  the  intent  that  image  gen- 
eration be  accomplished  utilizing  a high- 
speed CPU  in  combination  with  a high-speed 
arithmetic  unit.  Further  study,  however, 
indicated,  that  by  putting  more  of  the  image 
generation  work  into  special-purpose  sub- 
systems, an  enormous  increase  in  speed  could 
be  realized.  Thus,  the  major  workload  of 
image  generation  has  been  moved  into  the 
special-purpose  subsystems  and  placed  under 
the  control  of  the  CPU.  Its  high-speed 
architecture  has  been  retained  not  only  to 
provide  the  speed  necessary  to  control  the 
arithmetic  and  special  circuitry,  but  also  to 
provide  the  versatility  for  further  system 
growth.  This  growth  capability  accommodates 
further  research  and  development  by  permitting 
the  simulation  of  additional  features. 

The  ability  to  permit  host  computer  func- 
tions to  reside  in  the  CPU  may  also  be  accomo- 
dated. This  ability  may  be  useful  to  permit 
stand-alone  applications,  to  relieve  the  host 
computer  of  its  workload,  or  even  to  provide 
look  ahead  computations  to  enhance  image  gen- 
erator output. 


Provide  up  to  1,000  "X"  intercepts/scan- 
line. 2K,  3K,  etc.  intercepts  may  be  handled 
by  expansion  of  hardware  buffer  size. 

Within  the  edge  capability,  provide  for 
unlimited  high  resolution  aircraft  and/or 
ground  vehicle  images. 

Display  point  light  sources  (10,000  min.) 

Be  limited  only  by  the  mass  storage 
device  for  those  portions  of  the  world  outside 
the  FOV. 

The  C0MPUTR0L  system  is  a special-purpose 
design  and  does  not  use  general-purpose  hard- 
ware other  than  standard  peripheral  equipment 
such  as  magnetic  tape  or  disc  units.  The 
special  design  includes  a CPU  which  may  be 
utilized  to  facilitate  implementation  of  new 
computational  algorithms  and  to  support  fur- 
ther CGI  development. 

The  attached  data  sheet  illustrates 
COMPUTROL's  image  processing  capability. These 
photographs  have  been  taken  from  a standard 
15"  525  line  TV  monitor  that  has  been  modified 
by  increasing  the  bandwidth  of  the  video  am- 
plifiers by  a factor  of  five  and  the  vertical 


"resolution"  to  1200  lines  by  a means  of  a 5:1 
interlace. 

Dulles  terminal,  as  viewed  from  the  cock- 
pit window  of  a taxiing  aircraft,  consists  of 
2,162  edges. 

The  simulated  view  of  a CONCORDE  in  the 
landing  configuration  consists  of  910  edges. 
Color  shading  capability  may  be  appreciated  by 
a close  examination  of  the  fuselage.  Motion 
of  control  surfaces  (wheels,  nose  section, 
flaps,  etc.)  is  updated  in  each  new  TV  frame. 

The  dining  room  scene  consists  of  9,130 
edges  and  is  representative  of  the  detail 
possible  with  the  16,000  edge  early  develop- 
ment system. 

PHYSICAL  CHARACTERISTICS 

The  CGI  system,  exclusive  of  displays 
and  input  terminal,  is  housed  in  a total  of 
four  vertical,  joined  cabinets  suitably  fin- 
ished and  covered.  Cable  access  is  from  the 
rear  and  circuit  board  access  is  from  the 
front. 

One  cabinet  contains  the  wire-wrap  logic 
boards  for  all  CPU,  processing,  controller, 
cache,  and  interface  functions  required  for 
the  projection  and  visible  surface  image 
generators. 

The  second  and  third  cabinet  similar  to 
the  first  houses  main  memory  (MOS)  mounted  on 
PC  cards,  interconnected  via  edge  connectors, 
mounted  on  a "mother  board"  chassis,  and  two 
dual-disc  drives.  Blowers  and  power  displays 
are  distributed  in  the  fourth  cabinet  to  sup- 
port system  requirements.  A basic  control 
panel  is  also  included. 

DISPLAY 

The  Cockpit  Display  System  is  mounted  on 
the  flight  compartment/motion  platform  and 
produced  a real-time,  through  the  windshield 
color  virtual  image  display.  The  system  can 
include  one  or  more  high- resolution  color  moni- 
tors and  associated  infinity  viewing  optical 
systems. 

The  display  monitor  is  a 1029  scan  line 
RGB  color  monitor  using  a high-resolution 
shadow  mask  CRT  such  as  the  RCA  1908P22.  The 
gun  and  phosphor  design  provides  a brightness 
of  77  foot  lamberts  for  1 ma.  of  beam  current. 
This  is  a high-resolution  tube  with  20  mils 
spacing  between  centers  of  adjacent  triads.  A 
very  fine  line  width  of  10  mils  is  obtained 
at  a 40%  amplitude  level  of  the  beam  current 
density  distribution  curve  for  200  micro  amp 
per  gun  of  anode  current.  It  is  the  fine-beam 
spot  capability  of  this  CRT  which  permits  full 
use  to  be  made  of  the  small  triad  spacing. 


The  selection  of  the  RCA  1908P22  is  based  on 
its  resolution,  high  brightness,  availability, 
and  the  fact  that  this  type  has  been  success- 
fully used  on  other  beamsplitter,  spherical 
mirror  systems. 

DETAILED  DESIGN  CAPABILITIES 

COMPUTROL  uses  a numerically  stored  model 
and  data  of  the  real  world  to  generate  “out 
the  window"  visual  scenes.  All  objects  in  the 
data  base  are  modeled  in  three  dimensions  so 
that  the  observer  can  move  about  at  will, 
throughout  the  playing  area,  with  no  restric- 
tion on  movement,  direction,  altitude,  or 
velocity. 

Features  of  aircraft  or  other  vehicles 
will  be  of  sufficient  detail  to  be  visually 
identifiable  at  a range  equal  to  the  range  in 
real  life.  As  range  decreases  the  motion  of 
control  surfaces  will  become  visible.  These 
surfaces  will  move  as  if  they  were  actually 
controlling  the  motion  of  the  aircraft. 

The  COMPUTROL  is  designed  to  interface 
with  multiple  terminals.  The  terminal  screen 
will  act  as  a window  into  a three-dimensional 
world  containing  representations  of: 

physical  objects  symbols 
graphic  images  surfaces 
physical  events 

The  representations  in  this  3D  world  may: 

be  solid,  liquid,  or  gas 
have  any  color,  brightness,  sheen,  trans- 
parency 

have  any  shape 
be  point  light  sources 
have  any  location,  orientation,  scaling, 
and  magnification 

All  parameters  in  the  definition  of  the 
world  and  all  parameters  in  the  specification 
of  the  “window"  into  that  world  may  be  dynamic 
in  real-time.  Allowed  functions  of  time  or  of 
user  input  include: 

Polynomial  functions,  i.e.  constant 
velocity,  constant  acceleration,  etc. 
Analytical  functions  such  as  sine  (kt), 
square  root  (kt),  ekt,  log  (kt),  1/log 
(kt),  etc. 

Numerical  functions 
Arbitrary  space/time  trajectories 

Object/Objects  interactions  include: 

Illumination  shading  of  object  surfaces 
Collision  dynamics  (i.e.,  collision  de- 
tection and  conservation  of  momentum 
and  energy) 

Object  lock-on  (i.e.,  two  objects  collid- 
ing will  stick  together  during  and 
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following  trajectory  — useful  when 
one  of  the  objects  is  under  the  con- 
trol of  a hand  or  joystick  input  de- 
vice) 

Planar  scalpeling  and  "window"  clipping 
(i.e.,  selective  cross-sectioning  of 
subsets  of  objects) 

IMAGE  GENERATION 

Scenes,  as  displayed  on  the  CRT,  are 
described  by  the  edges  of  surfaces.  The  sys- 
tem has  the  capability  of  displaying  30,000 
edges.  A minimum  of  10,000  colored  point 
light  sources  are  available  for  use  in  the 
night  scene. 

The  surfaces  can  be  planar,  cylindrical, 
or  spherical  because  of  the  system's  ability 
to  "smooth  shade"  curved  surfaces.  The 
30,000  edges  is  a limitation  on  display  de- 
tail and  not  on  gaming  area  complexity.  There 
is  virtually  no  limit  to  the  number  of  edges 
in  the  gaming  area  data  base.  The  computer 
system  discards  edges  not  visible  by  virtue 
of  being  out  of  t ie  field  of  view  of  the 
pilot  or  ones  describing  hidden  surfaces. 
Another  feature  of  the  system  is  the  ability 
to  select  the  sun  angle.  This  will  be  evi- 
denced by  different  sides  of  buildings, 
hills,  etc.  being  in  shadow. 

As  the  pilot  nears  touchdown,  his  land- 
ing lights  will  illuminate  the  runway  ahead 
of  him  exactly  as  in  real  life.  The  area 
illuminated  by  each  light  is  computed  sep- 
arately to  produce  the  correct  presentation 
for  any  combination  of  lights  being  used. 

The  visual  system  will  accurately  depict 
the  simulator's  computer  flight  space  with 
sufficient  realism  and  definition  to  induce 
proper  psychological  motivation  and  provide 
capability  for  rigorous  pilot  training.  It 
will  operate  with  accuracy  and  response  so  as 
to  preclude  the  possibility  of  perceptible 
disagreement  between  the  visual  display  and 
other  parts  of  the  simulation,  including  any 
motion  cues  and  instrument  indications,  over 
the  entire  range  of  specified  operation.  The 
system  will  permit  the  aircrew  to  determine 
attitude,  altitude,  relative  attitude  rates, 
and  relative  velocity  by  reference  to  terrain 
and  horizon  features  in  the  same  manner  as  in 
an  actual  aircraft  under  visual  flight  con- 
ditions. 

Real  world  six-dimensionAl  changes  will  be 
simulated  in  size  and  perspective  realisti- 
cally and  accurately  in  real-time  with  re- 
spect to  aircraft  under  visual  flight  con- 
ditions. The  system  will  provide  realistic 
visual  simulation  of  the  taxi,  landing,  and 
takeoff  phases  of  flight  as  well  as  all 
flight  maneuvers.  All  maneuvers  will  be 
visually  current.  The  visual  system  will 


properly  display  attitude  and  flight  path 
characteristics  that  are  associated  with  side- 
slip and  crabbing  maneuvers  during  crosswind 
landings  and  general  maneuvering  flight. 

The  visual  system  will  portray  all  scene 
landing  in  the  field  of  view  in  their  proper 
perspective.  The  scene  elements  will  be  dis- 
played with  respect  to  their  modeled  coordi- 
nates to  within  three  arc-minutes  positional 
accuracy  as  seen  from  the  crewmember's  seated 
position. 

The  visual  system  will  track  the  simu- 
lator's computer  flight  position  to  within 
one-half  foot  and  its  computer  flight  attitude 
to  within  three  arc -minutes  for  the  duration 
of  the  flight.  Any  modeled  point  will  be 
locatable  to  within  one  inch  in  the  data  base. 

FUNCTIONAL  OPERATION  OF  THE  CGI  SYSTEM 

The  author  language  is  divided  into  an 
atom  language,  and  an  object/world  language. 

The  atom  language  permits  the  creation  of 
"primitive"  objects  out  of  xyz  data,  such 
primitive  objects  being  termed  atoms.  The 
object/world  language  permits  the  modification 
or  building  of  more  complex  objects/worlds  out 
of  other  objects  and  atoms. 

Worlds,  objects,  and  atoms,  portions 
thereof  and  operations  thereupon,  may  be  given 
symbolic  names  (labels)  for  their  construction 
and  manipulation. 

The  basic  library  contains  standard  two- 
and  three-dimensional  atoms  such  as  cubes, 
spheres,  cylinders,  wedges,  circles,  squares, 
and  triangles.  Special  atoms  are  created  for 
such  items  as  an  airplane  aileron  when  it  can- 
not be  adequately  represented  by  combining 
standard  shapes.  Once  created,  the  atom  is 
added  to  the  library  and  is  available  for  use 
wherever  needed.  The  atoms  are  created  in  one 
or  more  of  the  following  ways: 

By  polygon  input  method 

By  contour  input  method 

From  other  sets  of  points  by  interactive 
and/or  analytical  techniques 

By  modification  of  another  atom,  its 
points,  contours,  etc. 

Additional  capabilities  are  techniques  to 
warp,  bend,  and  cut.  Logical  functions  be- 
tween atoms  (volume  common  to  two  atoms  defines 
a third  atom,  etc.)  along  with  parametric 
specifications  of  an  atom  are  also  included. 

Objects  or  worlds  may  be  created  by 
assembling  them  out  of  transformations  of  one 
or  more  objects  or  atoms.  Allowable  trans- 
formations on  objects  are: 

Translation  (move  in  xyz)  Size  scaling 
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Surface  or  point  recoloring  (the  color 
within  a surface  is  a liner  inter- 
polation from  the  color  at  the  edges 
defining  the  surface) 

Rotation  (about  xyz) 

IMAGE  HANDLING 


Alphanumeric  CGI  and  keyboard 

Mass  storage  disc  drives  and  controllers 

Digitizing  tablet 

Card  reader/punch 

Operator's  monitor 

Printer  Teletype 

Link  with  the  host  computer 


An  image  is  encoded  by  a set  of  edges. 

An  edge  is  a real  or  an  imaginary  line  to  the 
right  of  which  is  displayed  a color.  The 
data  word  which  defines  an  edge  contains  such 
characteristics  as  color,  hue,  saturation, 
brightness,  and  magnitude  in  xyz  space.  There 
are  many  atoms  which  could  be  described,  but 
let  us  explore  the  cube  as  an  example. 


A polygon  of  arbitrary  shape  and  color 
is  represented  by  these  edges.  Figure  2 de- 
picts a perspective  of  a cube  represented  by 
nine  edges.  Because  the  color  may  vary  1 in- 
era  ly  along  edges  bounding  the  polygon  and 
lineraly  in  x between  edges,  a polygon  can 
also  represent  a portion  of  a curved  surface. 
By  means  of  these  polygons,  both  flat  and 
curved  surfaces  can  be  represented  in  any 
illumination  environment. 

When  objects  are  occulted  by  other  ob- 
jects, new  edges  are  created  called  "inter- 
section edges."  These  intersection  edges 
are  created  in  real-time  and  do  not  reduce 
the  basic  30K  edge  capability  of  the  system. 
They  are,  therefore,  not  counted  as  additional 
visible  edges. 


The  mass  storage  discs  are  arranged  into 
libraries  and  entries  within  libraries.  A 
library  is  generally  a logically  related  col- 
lection of  entries  such  as  object  codes  for 
programs,  atom  descriptions  and  compiler  out- 
put listings  to  be  printed  or  displayed  via 
CRT,  etc.  Libraries  and  entries  may  be  cre- 
ated, deleted,  or  modified  either  under  pro- 
gram control  or  under  user  control  via  inter- 
active commands.  There  is  no  restriction 
upon  the  number  or  size  of  libraries  permitted 
other  than  the  physical  restriction  of  disc 
storage  size.  A library  may  contain  either 
fixed- length  data  block  entries  for  random 
access  of  variable  length  data. 

Using  the  CGI  system  to  advantage,  the 
programmer  may  do  such  things  as: 

Delete  an  entry  from  a library 
Delete  an  entire  library  from  a disc 
Copy  an  entry  from  one  library  to  another 
Copy  a library  of  entries  to  another  disc 
Rename  an  entry  or  library 
Merge  two  libraries  together 
Load  a library  from  another  medium,  e.g. 
card  reader 

Copy  an  entry  or  entire  library  to  anoth- 
er medium,  e.g.  card  punch  or  printer 
List  the  names  of  library  entries,  li- 
braries on  a disc  or  discs  currently 
on  the  system 


Vie  foresee  that  a later  generation  of 
COMPUTROL  will  provide  the  capability  of 
computing  60K  or  100K  potentially  visible 
object  edges  and  displaying  them  in  any  single 
window  or  combination  of  windows  throughout 
the  FOV. 

DATA  INPUT 

Data  entry  sequence  is  used  by  the 
modeler  to  create  and  manipulate  descriptions 
of  two-  and  three-dimensional  objects  and 
display  real-time  perspective  views  of  these 
objects  at  his  monitor.  The  major  software 
components  involved  in  managing  the  data 
are  described  o?low. 

OPERATING  SYS  I EM 

The  monitor  program  handles  all  physical 
input/output  requirements  of  the  CGI  software. 
These  input/output  devices  include: 

Visual  display  channels 


TEXT  EDITOR 

The  text  editor  allows  a user  to  create, 
update  and  store  text  in  a general  form.  Each 
collection  of  text  is  stored  by  name  in  a disc 
library  and  may  be  retrieved  at  any  time  for 
subsequent  inspection  or  modification.  While 
the  text  may  be  any  arbitrary  sequence  of 
symbols  in  general,  more  specifically  it  will 
be  used  to  contain  the  descriptions  or  atoms 
and  objects  created  by  a modeler.  This  text 
may  be  initially  entered  by  using  a keyboard 
and  CRT  terminal,  by  using  a card  reader  as 
input  or  by  using  an  interactive  digitizing 
tablet  program  which  generates  text  as  its 
output.  The  modeler  specified  point  coordi- 
nate locations  and  point  linkages,  with  the 
program  generating  the  modeling  language  state- 
ments as  if  the  modeler  had  typed  these  state- 
ments directly.  Thus,  subsequent  keyboard 
modifications  to  the  data,  such  as  the  append- 
ing of  comments  of  other  features,  are  permit- 
ted, regardless  of  the  original  source  of  the 
input  text. 
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ATOM  COMPILER 


The  atom  compiler  processes  a user's 
text  description  of  the  three-dimensional 
structure  of  an  atom  and  generates  an  encoded 
description  of  the  atom  for  subsequent  pro- 
cessing by  the  image  generator.  The  result- 
ant compiler  atom  is  stored  by  name  in  a disc 
library  for  later  use  in  constructing  objects. 
As  delivered,  the  system  library  will  con- 
tain a wide  assortment  of  atoms  reflecting 
geometric  shapes  and  other  special  shapes 
encountered  in  modeling  the  data  base.  There 
is  no  limit  imposed  by  the  CGI  hardware  de- 
sign on  how  many  additional  atoms  and  objects 
can  be  defined  by  the  user.  Optionally,  a 
listing  file  is  produced  by  the  compiler  con- 
taining the  source  input  statements,  a sorted 
cross-reference  list  of  all  mnemonic  names 
used  in  the  atom  description  text  and  error 
diagnostic  messages  if  errors  were  encounter- 
ed during  compilation.  This  listing  file  may 
be  examined  on-line  using  the  text  editor, 
printed  on  a hard  copy  device,  retained  on 
disc  for  later  reference  or  deleted  from  its 
listing  library. 

OBJECT  COMPILER 

The  object  compiler  processes  text  des- 
cribing a collection  of  previously  defined 
atoms  and  objects  and  creates  one  or  more  new 
more  complex  objects  from  them.  The  input 
text  identifies  a set  of  atoms  and  objects  by 
their  names  and  specified  such  things  as  the 
color,  size,  and  location  of  each  with  re- 
spect to  the  others.  The  resultant  compiled 
object  is  stored  by  name  in  the  specified 
disc  library  for  subsequent  viewing  or  use  in 
constructing  other  objects.  As  with  the  atom 
compilation  process,  a listing  may  be  pro- 
duced by  the  object  compiler. 

WORLD  COMPILER 

The  world  compiler  prepares  a group  of 
objects  for  processing  by  the  image  generator. 
A world  consists  of  an  object  and  a set  of 
viewing  parameters.  The  viewing  parameters 
define  such  things  as  the  color  and  bright- 
ness of  the  background  area,  the  location  and 
brightness  of  the  sun  and  so  forth.  The 
world  compiler  is  also  responsible  for  seg- 
menting large  logically  related  sets  of  ob- 
jects into  "sub-worlds"  and  organizing  link- 
ages among  these  various  sub-worlds  so  that  a 
user  may  "roam"  through  the  world,  not  all  of 
which  can  be  held  in  the  image  generator's 
main  memory  at  once.  The  compilation  process 
is  in  no  way  sensitive  to  the  number  of  image 
generator  channels  to  be  used  in  displaying 
the  world. 


controls  the  operation  of  the  various  image 
generation  hardware  units.  It  first  performs 
parameter  substitution  of  all  numeric  para- 
meters of  a given  world  which  were  left  un- 
specified at  compilation  time.  These  para- 
meters are  supplied  by  the  modeler's  control 
devices  or  by  the  digital  computation  system, 
whichever  is  controlling  the  image  generator. 
After  parameter  substitution,  the  resultant 
compiler  world  is  treated  as  a command  list 
which  causes  the  image  generator  control  pro- 
gram to  command  the  individual  special-purpose 
hardware  units  to  transform  the  data  list  por- 
tion of  the  compiler  world  into  a set  of  pro- 
jected edges  in  the  user's  CRT  screen  domain. 
Finally,  the  projected  edges  are  transformed 
into  visible  edges  by  commanding  the  arith- 
metic units  of  the  visible  surface  processor. 

The  modeler  user  creates  his  world  from 
atoms  and  objects  as  detailed  in  the  preceed- 
ing  paragraphs.  He  combines  objects  and  atoms 
to  make  other  objects  and  scenes.  This  is 
done  in  the  coordinates  of  a "scratch"  world 
so  that  he  can  monitor  his  progress  without 
the  distraction  of  other  objects  which  might 
occult  the  object  being  worked  on.  When  com- 
plete, the  objects  are  given  real  world  coor- 
dinates and  orientation  to  place  them  properly 
in  the  playing  area  data  base.  It  is  also 
possible  to  remove  any  object  from  the  data 
base  to  the  scratoh  world  for  making  modifica- 
tions or  corrects. 

ATOM  LIBRARY 

The  library  of  standard  faces,  objects, 
and  models  which  comprise  the  atoms  and  ob- 
jects of  the  ATS  author  language  has  been  il- 
lustrated in  Figure  3.  The  ability  to  gener- 
ate new  atoms  and  objects  is  only  limited  by 
the  modeler's  imagination.  The  various  meth- 
ods of  constructing  the  basic  shapes  have 
already  been  described  with  illustrations  of 
the  variations  possible  shown  in  Figure  4. 


CUSC  SPHERE  CYIIROM 


SOUME  CIRCLE  n»tSE*T*T!V{ 

STMOMO  ATOMS 


IMAGE  GENERATOR  CONTROL  n,Ur.  3. 

The  image  generator  control  software 
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"world"  environment  requires  the  modeling  of 
terrain. 

GENERAL  TERRAIN 


Figure  4. 


These  atoms  were  specially  developed  to 
conform  to  the  real  world  atoms  of  the  Dulles 
Airport  complex. 

In  Figure  5 we  model  a car.  A basic  car 
"cube"  atom  has  been  extended  and  flattened 
to  represent  the  car  body.  A similar  atom  is 
added  to  >-epresent  the  top.  Other  modifica- 
tions of  the  basic  shape  produces  a car  image 
with  recognizable  detail.  The  modifications 
may  be  refined  until  the  car  has  the  fidelity 
of  an  artist's  rendering  of  a car. 


TERRAIN  GENERATION 


We  have  discussed  the  users  modeling  of 
various  objects.  To  place  the  objects  into  a 


I/O  equipment  will  be  used  to  input  raw 
data  from  constant  elevation  contours  of  topo- 
graphical maps.  Initial  data  reduction  would 
be  accomplished  by  techniques  that  monitor 
distances  and  radii  of  curvature  between  suc- 
cessive points  along  each  contour.  At  this 
stage,  color  data  can  be  Introduced;  a point 
that  Is  not  explicitly  tagged  with  color  data 
will  be  automatically  assigned  that  color  de- 
rived by  Interpolation  as  a function  of  its 
fractional  distance  between  two  points  that 
are  tagged  with  color  Information. 

HIGH- RESOLUTION  TERRAIN 


For  high-resolution  (close  up)  modeling, 
the  data  would  be  passed  through  a software 
package  that  connects  successive  contours  with 
triangular  facets.  The  technique  attempts  to 
optimize  the  selection  of  the  triangular  fac- 
ets by  minimizing  the  volume  of  the  resultant 
general  polyhedron  that  is  thus  formed.  There 
is  no  restriction  that  contour  lines  be  convex. 

MEDIUM-RESOLUTION  TERRAIN 


The  same  process  can  be  used  at  lower 
resolution  by  further  reducing  the  number  of 
redundant  data  points  along  a contour  and  by 
considering  fewer,  more  widely  separated  (in 
elevations)  contours. 

LOW- RES0L0TI0N  TERRAIN 


An  alternative  technique  that  could  be 
used  for  relatively  distant  terrain  would  be 
to  pass  the  initially  reduced  contour  data 
through  a software  package  that  would  perform 
a Fourier  analysis  of  the  surface;  that  is,  a 
set  of  parameters  would  be  generated  that 
would  permit  the  generation  of  a continuous 
analytical  surface  that  approximates  the 
original  terrain  surface.  For  lower  resolu- 
tions, the  higher  Fourier  coefficients  would 
be  excluded  from  the  computation  of  the  analy- 
tical surface. 

HIGH- RESOLUTION  DETAIL 


The  occurence  of  a significant  geographi- 
cal feature,  such  as  an  isolated  mountain  sur- 
rounded by  relatively  flat  field,  will  be 
handled  by  creating  them  separately  from  the 
general  terrain  and  placing  them  as  objects  in 
the  gaming  region. 

LAKES 


Lakes  (regions  of  constant  elevation)  lend 
themselves  nicely  to  the  constant  elevation 
contour  technique  since  the  shape  of  the  lake 
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should  be  independent  of  the  surrounding 
higher  elevation  terrain. 

PLACEMENTS  OF  OBJECTS  ON  TERRAIN 

Software  will  permit  the  modeler 
automatically  to  place  objects  at  (x.y.) 
positions  on  the  terrain.  This  is  particu- 
larly important,  for  example,  for  the  auto- 
matic placement  of  a 2-0  type  of  object  such 
as  a road  or  a river,  on  top  of  the  faces  of 
the  terrain  contour. 

DESIRABLE  FEATURES 

0CCULTATI0N 

For  any  given  viewpoint,  the  CGI  system 
will  generate  the  appropriate  perspective 
between  overlapping  surfaces.  As  conflicts 
between  overlapping  surfaces  are  resolved, 
intersection  edges  are  created  for  display  of 
proper  occultation  relationships. 

COLOR 

The  CGI  system  generates  image  coded  in 
nine  bits  for  each  of  the  red,  green,  and 
blue  channels  of  the  TV  display.  This  cor- 
responds to  512  intensity  levels  for  any 
color.  The  data  base  is  stored  by  specifying 
for  each  object  its  color  in  the  visible 
spectrum. 

CURVED  OBJECT  SIMULATIONS 

The  CGI  system  provides  realistic  curved 
object  simulation  by  assigning  color  to 
vertices  and  lineraly  varying  the  color  be- 
tween the  vertices  and  along  the  scanline 
between  edge  intersection.  Color  interpola- 
tion solutions  are  performed  in  parallel  with 
the  decoding  of  edges  and  do  not  affect  the 
linear  geometric  processing  or  display  cap- 
ability of  the  image  generator. 

PROGRAMMABLE  FIELD  OF  VIEW 

The  field  of  view  for  each  of  the  win- 
dows is  completely  programmable.  Thus,  for 
any  given  window  the  FOV  need  not  be  defined 
as  that  required  to  completely  fill  the  win- 
dow but  instead  can  be  that  FOV  defined  by 
any  portion  of  the  window.  This  is  possible 
since  the  window/clipping  parameters  are  pro- 
grammable and  can  be  altered  or  varied  as 
described. 

SYSTEM  OVERLOAD 

Overloads,  caused  by  excessive  edge 
capacity  in  a computed  scene,  is  eliminated 
by  employing  frame- to- frame  coherence  moni- 
toring. This  type  of  monitoring  senses  the 
number  of  visible  edges  in  a scene  and  uti- 
lizes this  number  to  cause  logical  simolifica- 


tions  of  the  world  when  overload  conditions 
are  approached.  Thus,  if  the  number  of  visi- 
ble edges  in  the  FOV  approaches  the  limit 
during  a frame  solution,  certain  objects  will 
be  simplified  to  assure  maintaining  the  edge 
count  below  the  maximum  capable  of  display  by 
the  system. 

If,  despite  the  logical  detection  and 
subsequent  object  simplification  an  overload 
does  occur,  the  system  design  is  such  that 
the  solution  is  completed  while  the  last 
solution  is  used  to  refresh  the  display. 

With  this  approach,  an  overload  will  not 
cause  picture  disintegration,  but  will  merely 
cause  a momentary  and  slight  delay.  The 
frame-to-frame  coherence  will  prevent  this 
overload  condition  from  persisting  and  of 
course  will  minimize  the  probability  of  its 
occurrence. 

POINT  LIGHTS 

The  CGI  system  permits  the  display  of  a 
minimum  of  10,000  colored  point  light  sources. 
Lights  can  be  modeled  whose  intensity  is 
constant  or  varies  in  accordance  with  slant 
range  and  time  (blinking).  There  are  no 
restrictions  on  placement. 

PERSPECTIVE  LIGHTS 

Perspective  lights  are  modeled  individu- 
ally by  the  author  language.  Several  differ- 
ent types  of  perspective  lights  are  possible 
including  omnidirectional,  such  as  taxiway 
lights  and  directional,  such  as  VASI  lights. 

In  addition,  a certain  number  of  the  per- 
spective lights  can  be  designated  as  light 
emitting.  These  will  emit  colored  light  with 
appropriate  color  mixing  and  surface  absorp- 
tion as  observed  in  a real  physical  environ- 
ment (a  blue  object  illumined  by  a red  light 
will  appear  black).  The  directional  lights 
are  modeled  with  hoods  such  that  the  cone  of 
visibility  may  be  specified. 

AERIAL  PERSPECTIVE 

"Aerial  perspective"  is  an  important 
ingredient  in  any  visual  presentation.  A 
certain  amount  of  visibility  limited  haze  or 
fog  almost  always  exists,  so  that  a scene 
without  it  lack  realism.  It  provides  an 
additional  altitude  cue  by  causing  the  hori- 
zon to  seem  lower  as  the  point  goes  higher, 
and  is  a factor  in  his  ability  to  judge  the 
range  of  targets  within  the  blanket  of  haze. 
The  fog  blanket  also  provides  a hiding  place 
for  ground,  or  low-flying  targets  and  so  is 
essential  in  combat  simulation  situations. 

The  visibility  restriction  is  simulated 
as  a low-lying  layer  of  haze,  fog,  or  smoke. 

It  is  densest  at  ground  level  and  thins  out 
with  height.  The  operator  selects  thickness 
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of  the  layer  as  well  as  the  density  at 
ground  level.  The  CGI  system  computes  the 
visibility  and  observed  color  of  each  com- 
ponent of  the  data  base  as  a function  of  the 
altitudes  of  the  object  and  the  observer,  and 
the  slant  distance  between  them. 

MOVING  MODEL  SIMULATION 

The  CGI  system  puts  no  limit  on  the 
number  of  items  in  the  data  base  that  can  be 
dynamic.  Likewise,  there  is  no  restriction 
on  the  degress  of  freedom  of  this  motion. 

Any  part  of  an  object  can  be  addressed  sep- 
arately so  that  incremental  movements  are 
also  simple,  that  is,  an  aileron  will  move 
with  the  airplane  but  also  rotates  about  its 
hinge  line. 

TEXTURE 


To  provide  surfaces  with  a realistic 
textured  appearance,  it  is  necessary  to  dis- 
play the  texture  pattern  with  the  same  per- 
spective processing  applied  to  object  or 
polygon  edges.  There  is,  therefore,  a need 
to  obtain  a perspective  of  each  discernable 
"characteristic"  of  a texture  pattern.  It  is 
appropriate  that  each  "characteristic"  be 
represented  and  processed  so  as  to  contain 
all  the  features  inherent  in  edge  representa- 
tion (perspective,  size,  brightness,  occulta- 
tion,  etc.) 

CLOUDS 


The  CGI  system  provides  for  operator 
selection  of  cloud  condition  ranging  from 
overcast  to  clear  skies.  The  bottom  of 
clouds  can  be  placed  at  any  height  above 
ground  level  while  cloud  tops  can  be  at  any 
altitude  above  the  bottoms. 

Clouds  are  treated  as  objects  and  are 
able  to  be  placed  where  needed  and  given 
appropriate  motions,  with  appropriate  shape 
and  size  functions  of  time.  Solid  cloud 
covers  and  scud  clouds  are,  therefore,  model- 
ed in  the  data  base. 

SUN  SIMULATION 

The  sun's  glare  and  horizon  glow  is 
modeled  by  two-dimensional  model  at  infinity. 
Since  shading  of  all  objects  is  computed  at 
execution  time,  the  location  of  the  sun  can 
be  dynamic.  It  can  thus  be  updated  every 
"frame"  along  with  its  effects  upon  the 
shading  of  the  world. 

When  the  sun  is  masked  by  clouds,  it 
results  in  a diffuse  illumination  of  the 
world.  This  kind  of  light  source,  which  is 
not  truly  directional.  Is  handled  by  altering 
the  brightness  of  fall-off  function  from  an 
angle  defining  the  direction  of  the  light 


1 


source  and  the  direction  of  the  surface.  The 
fall-off  function  is  a slower  fall-off  func- 
tion than  normally  encountered  with  a point 
light  source  at  infinity. 

SUMMARY 


In  recent  years,  we  have  come  to  realize 
that  physiological  motion  cues  are  but  one 
part  of  a total  simulation  experience.  It 
must  be  augmented  by  the  "visual"  cue  which 
appears  to  have  become  accepted  as  the  more 
important  aspect  of  the  two.  In  the  past,  we 
have  seen  a transition  of  Visual  System  tech- 
niques from  Film  Projection  to  Terrain  Model 
Boards  with  current  enthusiasm  for  Computer 
Generated  Image  Systems.  In  the  competitive 
struggle  for  increased  realism,  the  COMPUTROL 
Computer  Generated  Image  System  has  come  of 
age  due  to  its  ability  to  dynamically  change 
scene  content  in  real-time.  Not  only  is 
there  freedom  of  eye  movement  around  a parti- 
cular CGI  model,  but  also  the  capability  to 
effect  a "complete"  change  of  gaming  area  in 
reasonable  time.  The  goal  of  our  research 
and  develooment  efforts  has  been  to  incor- 
porate current  "chip"  technology,  resulting 
in  a new  computer  design  capable  of  high- 
speed data  handling.  The  objective  was  to 
create  a full-color  raster  scan  simulation  of 
a three-dimensional  real  world.  We  recognize 
that  the  product  of  our  research  and  develoD- 
ment  efforts  should  match  the  following  cri- 
teria. 

Resolution  and  fidelity  of  the  final 
scene  should  closely  equate  to  that 
of  optical  or  high-quality  film 
systems. 

Minimal  power  requirements  for  economi- 
cal operation. 

Inherent  capability  to  quickly  change 
entire  gaming  areas. 

Gaming  data  size  sufficient  to  permit 
extended  flight  simulation  without 
reaching  the  limit  of  stored  data. 

High-speed  computation  to  permit  dynamic 
movement  of  objects  within  the  dis- 
played scene,  e.g.  moving  vehicle 
(land,  sea,  or  airborne). 

System  design  sophistication  to  permit 
miniaturization  of  the  entire  system. 

Reasonable  purchase  price  and  cost  of 
ownership.  Affordable  by  a majority 
of  customers. 

We  feel  confident  that  our  research 
efforts  have  met  these  seven  basic  criteria. 

A 16,000  edge  full-color  static  demonstrator 
has  been  developed.  Capability  to  generate 
high-definition  pictures  with  variable  sun 
angle,  smooth  shading,  and  the  occultation  of 
hidden  surfaces  has  been  proven.  Occulting 
of  objects  through  the  visible  edge  solution 
guarantees  nonbleed  through  of  hidden  lines. 
Another  feature  of  the  COMPUTROL .windows  look 
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like  windows  rather  than  holes  in  buildings. 
The  transition  to  and  from  clouds  is  also 
much  more  convincing. 

System  architecture  and  special  algor- 
ithms are  used  to  permit  the  incorporation 
of  texturing  with  respect  to  trees,  grass, 
clouds,  and  water.  The  key  ingredient  of 
the  system  design  is  the  "atom"  philosophy 
of  geometric  forms  used  as  building  blocks 
for  the  total  scene  model.  The  basic  forms 
are  stretched,  squashed,  lengthened  and/or 
added  to  other  geometric  forms  to  develop 
a particular  scene.  The  modeling  and  use  of 
atoms  is  performed  in  the  off-line  mode 
where  there  is  the  ability  to  generate  and 
modify  scenes  as  well  as  to  edit  and  assem- 
ble programs.  With  the  exception  of  flight 
data  or  any  other  vehicle  interface  which 
is  resident  within  a simulator  "host  compu- 


ter," the  visual  system  is  a self-contained 
digital  image  generator  with  bulk  storage 
of  specific  geographic  areas.  The  current 
design  has  the  capability  of  displaying 
30,000  edges.  The  surfaces  can  be  planer  or 
spherical  due  to  the  systems'  ability  to 
smooth-shade  curved  surfaces.  There  is 
virtually  no  limit  to  the  number  of  edges 
that  can  be  contained  within  the  gaming 
area  data  base.  The  special-purpose  CPU 
used  with  the  image  generation  system  has 
been  especially  developed  to  provide  the 
computational  speed  necessary  within  the 
total  system.  Risk  normally  associated  with 
a specially  designed  computer  system  has  been 
substantially  reduced  due  to  the  use  of 
standard  MOS  chips  and  straight-forward 
computer  architecture.  The  resulting  CGI 
technology  promises  to  set  the  standard  for 
visual  simulators  for  many  years  to  come. 
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